Skip to content

fix(app): hydrate source fields for raw SQL tiles in all chart rendering contexts #2503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pulpdrew wants to merge 1 commit into from
Closed

fix(app): hydrate source fields for raw SQL tiles in all chart rendering contexts #2503

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .changeset/fix-raw-sql-source-hydration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
'@hyperdx/app': patch
---

fix: hydrate source metadata for raw SQL tiles in all chart rendering contexts

Raw SQL tiles referencing `$__sourceTable` or `$__filters` macros now correctly
resolve the source's table metadata regardless of the rendering context
(dashboards, notebooks, chart explorer). Previously, only the Dashboard Tile
component hydrated the `from`, `metricTables`, and other source-dependent fields
before query execution; other contexts passed the saved config directly, causing
macro expansion to fail with "Macro '$__sourceTable' requires a source to be
selected" despite the source being configured.
72 changes: 57 additions & 15 deletions packages/app/src/hooks/useChartConfig.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { pick } from 'lodash';
import {
chSqlToAliasMap,
ClickHouseQueryError,
Expand All @@ -22,7 +23,11 @@ import {
BuilderChartConfigWithOptDateRange,
ChartConfigWithDateRange,
ChartConfigWithOptDateRange,
getSampleWeightExpression,
isLogSource,
isMetricSource,
QuerySettings,
TSource,
} from '@hyperdx/common-utils/dist/types';
import {
useQuery,
Expand All @@ -41,6 +46,35 @@ import { generateTimeWindowsDescending } from '@/utils/searchWindows';

import { useMVOptimizationExplanation } from './useMVOptimizationExplanation';

/**
* Hydrate a raw SQL chart config with source metadata fields required for
* macro expansion ($__sourceTable, $__filters). The saved config only stores
* the source ID; the runtime fields (from, metricTables, etc.) must be merged
* from the fetched Source document before the config is rendered.
*
* Mirrors the hydration done in the Dashboard Tile useEffect
* (DBDashboardPage.tsx) so that all chart rendering contexts — dashboards,
* notebooks, chart explorer — resolve macros identically.
*/
export function hydrateRawSqlConfigFromSource<
T extends ChartConfigWithOptDateRange,
>(config: T, source: TSource | undefined): T {
if (!isRawSqlChartConfig(config) || !config.source || !source) {
return config;
}
return {
...config,
...pick(source, [
'implicitColumnExpression',
'useTextIndexForImplicitColumn',
'from',
]),
...(isLogSource(source) ? { bodyExpression: source.bodyExpression } : {}),
metricTables: isMetricSource(source) ? source.metricTables : undefined,
sampleWeightExpression: getSampleWeightExpression(source),
};
}

interface AdditionalUseQueriedChartConfigOptions {
onError?: (error: Error | ClickHouseQueryError) => void;
/**
Expand Down Expand Up @@ -295,26 +329,31 @@ export function useQueriedChartConfig(
id: config.source,
});

const hydratedConfig = hydrateRawSqlConfigFromSource(config, source);

const query = useQuery<TQueryFnData, ClickHouseQueryError | Error>({
// Include enableQueryChunking in the query key to ensure that queries with the
// same config but different enableQueryChunking values do not share a query
queryKey: [
config,
hydratedConfig,
options?.enableQueryChunking ?? false,
options?.enableParallelQueries ?? false,
],
// TODO: Replace this with `streamedQuery` when it is no longer experimental. Use 'replace' refetch mode.
// https://tanstack.com/query/latest/docs/reference/streamedQuery
queryFn: async context => {
// PromQL queries go through the Prometheus API route, not ClickHouse proxy
if (isPromqlChartConfig(config) && config.dateRange) {
const [startDate, endDate] = config.dateRange;
if (isPromqlChartConfig(hydratedConfig) && hydratedConfig.dateRange) {
const [startDate, endDate] = hydratedConfig.dateRange;
const startSec = startDate.getTime() / 1000;
const endSec = endDate.getTime() / 1000;

// Convert HyperDX granularity ("5 minute") to Prometheus step ("300s")
let stepStr = '60s';
if (config.granularity && config.granularity !== 'auto') {
if (
hydratedConfig.granularity &&
hydratedConfig.granularity !== 'auto'
) {
const granToSec: Record<string, number> = {
'15 second': 15,
'30 second': 30,
Expand All @@ -329,17 +368,17 @@ export function useQueriedChartConfig(
'12 hour': 43200,
'1 day': 86400,
};
stepStr = `${granToSec[config.granularity] ?? 60}s`;
stepStr = `${granToSec[hydratedConfig.granularity] ?? 60}s`;
}

const resp = await prometheusApi.queryRange({
query: config.promqlExpression,
query: hydratedConfig.promqlExpression,
start: startSec,
end: endSec,
step: stepStr,
connectionId: config.connection,
database: config.from?.databaseName ?? 'default',
table: config.from?.tableName ?? 'otel_metrics_ts',
connectionId: hydratedConfig.connection,
database: hydratedConfig.from?.databaseName ?? 'default',
table: hydratedConfig.from?.tableName ?? 'otel_metrics_ts',
});

if (resp.status !== 'success' || !resp.data) {
Expand Down Expand Up @@ -394,7 +433,8 @@ export function useQueriedChartConfig(
};
}

const optimizedConfig = mvOptimizationData?.optimizedConfig ?? config;
const optimizedConfig =
mvOptimizationData?.optimizedConfig ?? hydratedConfig;
const query = queryClient
.getQueryCache()
.find({ queryKey: context.queryKey, exact: true });
Expand Down Expand Up @@ -477,18 +517,20 @@ export function useRenderedSqlChartConfig(
id: config.source,
});

const hydratedConfig = hydrateRawSqlConfigFromSource(config, source);

const query = useQuery({
queryKey: ['renderedSql', config],
queryKey: ['renderedSql', hydratedConfig],
queryFn: async () => {
const optimizedConfig = mvOptimizationData?.optimizedConfig ?? config;
const optimizedConfig =
mvOptimizationData?.optimizedConfig ?? hydratedConfig;
const query = await renderChartConfig(
optimizedConfig,
metadata,
source?.querySettings,
);
const sql = parameterizedQueryToSql(query);
// sql-formatter can't handle prometheusQuery() / CTE syntax in PromQL queries
if (isPromqlChartConfig(config)) {
if (isPromqlChartConfig(hydratedConfig)) {

@greptile-apps greptile-apps Bot Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Dropped sql-formatter comment

The comment explaining why PromQL queries skip format() was removed when config was renamed to hydratedConfig. The formatter still cannot handle prometheusQuery() / CTE PromQL syntax, so the guard still needs the explanation. Without it, future readers may not understand why PromQL is returned early.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude Code Fix in Conductor Fix in Cursor Fix in Codex

return sql;
}
return format(sql);
Expand All @@ -498,7 +540,7 @@ export function useRenderedSqlChartConfig(
enabled &&
!isLoadingMVOptimization &&
!isSourceLoading &&
!isPromqlChartConfig(config),
!isPromqlChartConfig(hydratedConfig),
});

return {
Expand Down
Loading