Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
1abd469
mark the top bar location
RyanL1997 Apr 2, 2025
f929d31
draft - trying to find index name through panels
RyanL1997 Apr 3, 2025
d73087e
demo experiment 1 for button on every panels
RyanL1997 Apr 4, 2025
52b8e5a
revert to clean start for embeddable_panel class
RyanL1997 Apr 4, 2025
fc874c6
revert plugin.tsx for the internal embeddable plugin
RyanL1997 Apr 4, 2025
2b8230b
poc phase 0 - setup the collector function for collecting panel meta-…
RyanL1997 Apr 7, 2025
fcf4a8b
poc phase 1 - implement hte fetch for getting the index pattern id of…
RyanL1997 Apr 7, 2025
b70e6f0
poc phase 2 - get the name of mv index
RyanL1997 Apr 8, 2025
9ecfa9b
poc phase 2 - get the datasource/database/given index name info
RyanL1997 Apr 8, 2025
083ad41
poc phase 3 - implement a dummy refresh button
RyanL1997 Apr 8, 2025
6d912c3
poc phase 4 - introduce http into the grid class but no server setup yet
RyanL1997 Apr 9, 2025
26e40d3
poc phase 5 - FIX THE BUTTON FOR TESTING INDEX MAPPING CALL AND SET U…
RyanL1997 Apr 9, 2025
fd5d56b
poc related - remove unnecessary comment
RyanL1997 Apr 9, 2025
74a177a
poc phase 6 - implement refresh query call but no polling
RyanL1997 Apr 9, 2025
c41b2dd
poc phase 7 - implement direct query hook in dashboard plugin
RyanL1997 Apr 15, 2025
9998922
poc phase 7 - using internal resolve index api for getting index name…
RyanL1997 Apr 15, 2025
62698c2
Progress proto
Swiddis Apr 16, 2025
dd4a285
CLEAN UP THE UNNECESSARY SERVER SETUP
RyanL1997 Apr 16, 2025
85c7a84
refactor - extract the logic for sync out of grid class
RyanL1997 Apr 16, 2025
abd4512
poc phase 8 - add index mapping fetch
RyanL1997 Apr 17, 2025
6d0a928
Progress proto v2
Swiddis Apr 17, 2025
46c8ed0
FIX SOME LINTING ERROR
RyanL1997 Apr 18, 2025
01a6b6d
Refactor - moving the EMR status utils into dq sync class
RyanL1997 Apr 18, 2025
40c2f79
Refactor - moving the sync related UI component into a seprate class
RyanL1997 Apr 18, 2025
22ee2b2
Lint - remove some unused imports
RyanL1997 Apr 18, 2025
fca5928
production phase 0 - add the implementation of feature flag dashboard…
RyanL1997 Apr 18, 2025
f1205ca
Add some EMR states comments
Swiddis Apr 18, 2025
df3718d
production phase 1 - add the mds support
RyanL1997 Apr 23, 2025
5e6c46b
Update to new sync UI
Swiddis Apr 25, 2025
14e563c
production phase 3 - implement the URL feature flag
RyanL1997 Apr 25, 2025
d0a08fd
production phase 4 - fix the resolve index for mds support
RyanL1997 Apr 26, 2025
fac529e
production phase 5 - enhance the souce checking logic (handle more th…
RyanL1997 Apr 29, 2025
83d7734
production phase 5 - enhance the source checking logic a bit to handl…
RyanL1997 Apr 29, 2025
d9b7c0c
production phase 6 - add some basic test cases for util classes first
RyanL1997 Apr 29, 2025
71ceeaf
production phase 7 - clean up the hook
RyanL1997 Apr 29, 2025
05ab96f
production phase 8 - rename the sync component as direct query instea…
RyanL1997 Apr 29, 2025
23fa5d0
minor cleanup
RyanL1997 Apr 29, 2025
499f4c2
production phase 9 - fix existing grid test and some broken snapshots
RyanL1997 Apr 30, 2025
614da99
production phase 9 - fix existing viewport tests
RyanL1997 Apr 30, 2025
d6c3ffd
production phase 10 - add test cases in dashboards grid test
RyanL1997 Apr 30, 2025
b953163
Merge branch 'main' into vended-d-refresh
RyanL1997 May 1, 2025
d5fe718
fix lint
RyanL1997 May 1, 2025
0a1d857
Merge branch 'main' into vended-d-refresh
RyanL1997 May 1, 2025
07a5a35
Merge branch 'main' into vended-d-refresh
RyanL1997 May 5, 2025
0d32437
fix ci - 0
RyanL1997 May 5, 2025
c554c1e
fix ci - 1 remove inline style
RyanL1997 May 5, 2025
e05b574
production phase 11 - setup scss instead of inline style
RyanL1997 May 5, 2025
37a8b83
production phase 12 - do not go into the logic for check when feature…
RyanL1997 May 5, 2025
77ce840
Merge branch 'main' into vended-d-refresh
RyanL1997 May 6, 2025
3556596
Add more test cases
RyanL1997 May 6, 2025
eed6a45
modify the header
RyanL1997 May 6, 2025
6517e60
remove the unhandled error in extract info function and update the test
RyanL1997 May 6, 2025
90ed23e
Merge branch 'main' into vended-d-refresh
RyanL1997 May 6, 2025
29a34fb
remove unnecessary comments
RyanL1997 May 6, 2025
074f50b
Changeset file for PR #9745 created/updated
opensearch-changeset-bot[bot] May 6, 2025
e747e4e
Changeset file for PR #9745 created/updated
opensearch-changeset-bot[bot] May 6, 2025
945b8bb
add more test coverage for grid class
RyanL1997 May 6, 2025
7e393d0
add more test coverage for polling
RyanL1997 May 6, 2025
9e62b5a
enhance the flint info fetching parsing logic
RyanL1997 May 7, 2025
887d126
Merge branch 'main' into vended-d-refresh
RyanL1997 May 7, 2025
68ada45
REUSE FRAMEWORKS IN DSM
RyanL1997 May 7, 2025
6ff52a3
remove unnecessary bundle
RyanL1997 May 7, 2025
a08af10
UPDATE THE DIRECT QUERY HOOK TESTS
RyanL1997 May 7, 2025
aa84467
fix the acc creation status check due to the outdated loading status
RyanL1997 May 7, 2025
8c831f9
enhance the window reload logic
RyanL1997 May 7, 2025
a68483a
fix header of test 2
RyanL1997 May 7, 2025
29bd5fd
Merge branch 'main' into vended-d-refresh
RyanL1997 May 7, 2025
5ed1d3a
josh - address i18n-0
RyanL1997 May 7, 2025
fc7383c
josh - address feature flag var renaming
RyanL1997 May 7, 2025
583a607
josh - extract feature flag check into a function
RyanL1997 May 7, 2025
36c11c7
josh - emr state renaming
RyanL1997 May 7, 2025
8118a6f
josh - apply suggestion - remove any for query type
RyanL1997 May 7, 2025
a12ea47
josh - apply suggestion - private sync function and add some comments
RyanL1997 May 7, 2025
0924714
josh - add lang parameter to accept more language types
RyanL1997 May 7, 2025
d0836dc
josh - add lang parameter to accept more language types and add some …
RyanL1997 May 7, 2025
d7f5b5c
josh - add some comment for extractIndexInfoFromDashboard
RyanL1997 May 7, 2025
06ca32d
josh - add i18n to min on ui
RyanL1997 May 7, 2025
135af2c
josh - remove any in ref param
RyanL1997 May 7, 2025
8ffc464
josh - refactor emr status map
RyanL1997 May 7, 2025
e0fd3ef
josh - add commment for wrapper DashboardViewportWithQuery
RyanL1997 May 7, 2025
be93742
josh - chaneg the return to null instead of unknown
RyanL1997 May 7, 2025
51a5d3f
josh - EXTRACT AS THE LOGIC IN GRID AS A SERVICE
RyanL1997 May 7, 2025
50d1932
josh - REVERT GRID TEST AND JUST ADD A CASE FOR SYNC
RyanL1997 May 7, 2025
0600777
josh - revert unnecessary changes in grid class
RyanL1997 May 7, 2025
faa5f90
enhance service and add tests
RyanL1997 May 7, 2025
a339e70
josh - relocate sync service under direct query sync dir
RyanL1997 May 7, 2025
222a364
josh - move render logic in grid to upper level and fix tests
RyanL1997 May 7, 2025
7f30454
josh - move the feature flag logic into service
RyanL1997 May 7, 2025
614edf7
josh - switch to add dsm as required bundle instead of plugin
RyanL1997 May 7, 2025
4054f9d
revert grid back
RyanL1997 May 7, 2025
57015d6
fix ft
RyanL1997 May 8, 2025
f6682de
relocate the sync dir
RyanL1997 May 8, 2025
edd48d2
WRAP VIEWPORT IN DSHBOARD CAONTAINER
RyanL1997 May 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions changelogs/fragments/9745.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
feat:
- [Integration] Vended Dashboards Synchronization #9745 ([#9745](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/9745))
1 change: 1 addition & 0 deletions src/plugins/dashboard/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import { schema, TypeOf } from '@osd/config-schema';

export const configSchema = schema.object({
allowByValueEmbeddables: schema.boolean({ defaultValue: false }),
directQueryConnectionSync: schema.boolean({ defaultValue: false }),
});

export type ConfigSchema = TypeOf<typeof configSchema>;
2 changes: 1 addition & 1 deletion src/plugins/dashboard/opensearch_dashboards.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,5 @@
"optionalPlugins": ["home", "share", "usageCollection"],
"server": true,
"ui": true,
"requiredBundles": ["opensearchDashboardsUtils", "opensearchDashboardsReact", "home"]
"requiredBundles": ["opensearchDashboardsUtils", "opensearchDashboardsReact", "home", "dataSourceManagement"]
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,10 @@
OpenSearchDashboardsReactContext,
OpenSearchDashboardsReactContextValue,
} from '../../../../opensearch_dashboards_react/public';
import { DirectQuerySyncProvider } from './direct_query_sync/direct_query_sync_context';
import { PLACEHOLDER_EMBEDDABLE } from './placeholder';
import { PanelPlacementMethod, IPanelPlacementArgs } from './panel/dashboard_panel_placement';
import { DashboardFeatureFlagConfig } from '../../plugin';

export interface DashboardContainerInput extends ContainerInput {
viewMode: ViewMode;
Expand Down Expand Up @@ -104,6 +106,9 @@
SavedObjectFinder: React.ComponentType<any>;
ExitFullScreenButton: React.ComponentType<any>;
uiActions: UiActionsStart;
savedObjectsClient: CoreStart['savedObjects']['client'];
http: CoreStart['http'];
dashboardFeatureFlagConfig: DashboardFeatureFlagConfig;
}

export type DashboardReactContextValue = OpenSearchDashboardsReactContextValue<
Expand All @@ -121,6 +126,9 @@

private embeddablePanel: EmbeddableStart['EmbeddablePanel'];
private readonly logos: Logos;
private hasRendered: boolean = false;
private renderCount: number = 0;
private instanceId: string;

constructor(
initialInput: DashboardContainerInput,
Expand All @@ -136,10 +144,22 @@
options.embeddable.getEmbeddableFactory,
parent
);
this.instanceId = uuid.v4();
console.log(`[DashboardContainer] Constructor called (instanceId: ${this.instanceId})`);

Check failure on line 148 in src/plugins/dashboard/public/application/embeddable/dashboard_container.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement
this.embeddablePanel = options.embeddable.getEmbeddablePanel(stateTransfer);
this.logos = options.chrome.logos;
}

public componentDidMount() {
console.log(`[DashboardContainer] componentDidMount (instanceId: ${this.instanceId})`);

Check failure on line 154 in src/plugins/dashboard/public/application/embeddable/dashboard_container.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement
}

public componentWillUnmount() {
console.log(`[DashboardContainer] Unmounting (instanceId: ${this.instanceId})`);

Check failure on line 158 in src/plugins/dashboard/public/application/embeddable/dashboard_container.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement
this.hasRendered = false;
this.renderCount = 0;
}

protected createNewPanelState<
TEmbeddableInput extends EmbeddableInput,
TEmbeddable extends IEmbeddable<TEmbeddableInput, any>
Expand Down Expand Up @@ -190,10 +210,6 @@
previousPanelState: DashboardPanelState<EmbeddableInput>,
newPanelState: Partial<PanelState>
) {
// TODO: In the current infrastructure, embeddables in a container do not react properly to
// changes. Removing the existing embeddable, and adding a new one is a temporary workaround
// until the container logic is fixed.

const finalPanels = { ...this.input.panels };
delete finalPanels[previousPanelState.explicitInput.id];
const newPanelId = newPanelState.explicitInput?.id ? newPanelState.explicitInput.id : uuid.v4();
Expand Down Expand Up @@ -235,20 +251,48 @@
}

public render(dom: HTMLElement) {
ReactDOM.render(
<I18nProvider>
<OpenSearchDashboardsContextProvider services={this.options}>
<DashboardViewport
key={this.id}
renderEmpty={this.renderEmpty}
logos={this.logos}
container={this}
PanelComponent={this.embeddablePanel}
/>
</OpenSearchDashboardsContextProvider>
</I18nProvider>,
dom
this.renderCount++;
console.log(

Check failure on line 255 in src/plugins/dashboard/public/application/embeddable/dashboard_container.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement
`[DashboardContainer] Rendering (instanceId: ${this.instanceId}, count: ${this.renderCount}, hasRendered: ${this.hasRendered})`
);

const DashboardContainerWrapper = () => {
return (
<I18nProvider>
<OpenSearchDashboardsContextProvider services={this.options}>
<DirectQuerySyncProvider
savedObjectsClient={this.options.savedObjectsClient}
http={this.options.http}
notifications={this.options.notifications}
isDirectQuerySyncEnabled={
this.options.dashboardFeatureFlagConfig.directQueryConnectionSync
}
container={this}
>
<DashboardViewport
key={this.id}
renderEmpty={this.renderEmpty}
logos={this.logos}
container={this}
PanelComponent={this.embeddablePanel}
/>
</DirectQuerySyncProvider>
</OpenSearchDashboardsContextProvider>
</I18nProvider>
);
};

if (!this.hasRendered) {
console.log(

Check failure on line 286 in src/plugins/dashboard/public/application/embeddable/dashboard_container.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement
`[DashboardContainer] Initial render with ReactDOM.render (instanceId: ${this.instanceId})`
);
ReactDOM.render(<DashboardContainerWrapper />, dom);
this.hasRendered = true;
} else {
console.log(

Check failure on line 292 in src/plugins/dashboard/public/application/embeddable/dashboard_container.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement
`[DashboardContainer] Skipping ReactDOM.render, already rendered (instanceId: ${this.instanceId})`
);
}
}

protected getInheritedInput(id: string): InheritedChildInput {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.dshDashboardGrid__syncBar {
margin-bottom: $euiSizeS;
margin-left: $euiSizeS;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import React from 'react';
import { render, screen, fireEvent } from '@testing-library/react';
import '@testing-library/jest-dom';
import { DashboardDirectQuerySync } from './dashboard_direct_query_sync';

describe('DashboardDirectQuerySync', () => {
const mockSynchronize = jest.fn();

it('renders sync info with refresh link when state is terminal', () => {
render(
<DashboardDirectQuerySync
loadStatus="success"
lastRefreshTime={Date.now() - 60000}
refreshInterval={2}
onSynchronize={mockSynchronize}
/>
);

expect(screen.getByText(/Data scheduled to sync every/i)).toBeInTheDocument();
expect(screen.getByText(/Sync data/i)).toBeInTheDocument();

fireEvent.click(screen.getByText(/Sync data/i));
expect(mockSynchronize).toHaveBeenCalled();
});

it('renders loading spinner when state is not terminal', () => {
render(
<DashboardDirectQuerySync
loadStatus="running"
lastRefreshTime={undefined}
refreshInterval={undefined}
onSynchronize={mockSynchronize}
/>
);

expect(screen.getByText(/Data sync is in progress/i)).toBeInTheDocument();
expect(screen.queryByText(/Sync data/i)).toBeNull();
});

it('handles missing lastRefreshTime and refreshInterval gracefully', () => {
render(
<DashboardDirectQuerySync
loadStatus="success"
lastRefreshTime={undefined}
refreshInterval={undefined}
onSynchronize={mockSynchronize}
/>
);

expect(
screen.getByText(
(content) => content.includes('Data scheduled to sync every') && content.includes('--')
)
).toBeInTheDocument();

expect(screen.getByText(/Sync data/i)).toBeInTheDocument();
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import React from 'react';
import { EuiCallOut, EuiLink, EuiLoadingSpinner, EuiText } from '@elastic/eui';
import { i18n } from '@osd/i18n';
import { DirectQueryLoadingStatus } from '../../../../../data_source_management/public';
import { EMR_STATES, intervalAsMinutes } from './direct_query_sync';
import './_dashboard_direct_query_sync.scss';

export interface DashboardDirectQuerySyncProps {
loadStatus?: DirectQueryLoadingStatus;
lastRefreshTime?: number;
refreshInterval?: number;
onSynchronize: () => void;
className?: string;
}

export const DashboardDirectQuerySync: React.FC<DashboardDirectQuerySyncProps> = ({
loadStatus,
lastRefreshTime,
refreshInterval,
onSynchronize,
className,
}) => {
console.log('DashboardDirectQuerySync: Rendering with props:', {

Check failure on line 28 in src/plugins/dashboard/public/application/embeddable/direct_query_sync/dashboard_direct_query_sync.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement
loadStatus,
lastRefreshTime,
refreshInterval,
className,
onSynchronize: '[Function]', // Simplified since onSynchronize is guaranteed to be a function
});

// If loadStatus is undefined, default to a non-terminal state to avoid errors
const state = loadStatus ? EMR_STATES.get(loadStatus)! : { ord: 0, terminal: false };
console.log('DashboardDirectQuerySync: Computed state=', state);

Check failure on line 38 in src/plugins/dashboard/public/application/embeddable/direct_query_sync/dashboard_direct_query_sync.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement

return (
<div className={className} data-test-subj="dashboardDirectQuerySyncBar">
{state.terminal ? (
<>
{console.log('DashboardDirectQuerySync: Rendering terminal state UI')}

Check failure on line 44 in src/plugins/dashboard/public/application/embeddable/direct_query_sync/dashboard_direct_query_sync.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement
<EuiText size="s">
{i18n.translate('dashboard.directQuerySync.dataScheduledToSync', {
defaultMessage: 'Data scheduled to sync every {interval}. Last sync: {lastSyncTime}.',
values: {
interval: refreshInterval ? intervalAsMinutes(1000 * refreshInterval) : '--',
lastSyncTime: lastRefreshTime
? `${new Date(lastRefreshTime).toLocaleString()} (${intervalAsMinutes(
new Date().getTime() - lastRefreshTime
)} ago)`
: '--',
},
})}

<EuiLink onClick={onSynchronize}>
{i18n.translate('dashboard.directQuerySync.syncDataLink', {
defaultMessage: 'Sync data',
})}
</EuiLink>
</EuiText>
</>
) : (
<>
{console.log('DashboardDirectQuerySync: Rendering in-progress state UI')}

Check failure on line 67 in src/plugins/dashboard/public/application/embeddable/direct_query_sync/dashboard_direct_query_sync.tsx

View workflow job for this annotation

GitHub Actions / Lint and validate

Unexpected console statement
<EuiCallOut size="s">
<EuiLoadingSpinner size="s" />

{i18n.translate('dashboard.directQuerySync.dataSyncInProgress', {
defaultMessage:
'Data sync is in progress ({progress}% complete). The dashboard will reload on completion.',
values: {
progress: state.ord,
},
})}
</EuiCallOut>
</>
)}
</div>
);
};
Loading
Loading