Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
e393965
init
mdroidian Apr 27, 2024
00aa560
config page, roam comments
mdroidian Apr 29, 2024
a323e7a
add githubsync
mdroidian Apr 30, 2024
bf89829
get comments
mdroidian May 1, 2024
f342f36
add comments header
mdroidian May 1, 2024
8fc4d29
multipele: observers fixed, add comment, get comments, sent page to g…
mdroidian May 3, 2024
b17b1ea
rebase
mdroidian May 9, 2024
79617c4
working state
mdroidian May 8, 2024
e953356
rebase
mdroidian May 9, 2024
bae6519
github file content encode fix
mdroidian May 9, 2024
f9f14a6
rebase
mdroidian May 9, 2024
402ada7
misc
mdroidian May 9, 2024
1b2bd93
use existing onClose()
mdroidian May 9, 2024
dab592a
use createConfigObserver instead of renderConfigPage
mdroidian May 9, 2024
32452f3
misc
mdroidian May 9, 2024
af67257
save repo to issue and pull from props when downloading comments
mdroidian May 10, 2024
91f3aba
better copy
mdroidian May 10, 2024
d8b5e26
add github api reference links
mdroidian May 10, 2024
8565186
add docs
mdroidian May 10, 2024
c79a736
link to docs from Readme
mdroidian May 10, 2024
d3d7c04
add comment cache (#254)
mdroidian May 14, 2024
82c8df9
better error messages
mdroidian May 14, 2024
6b6b52f
change Comments Block from reference to inline query
mdroidian May 14, 2024
f899fee
misc PR Feedback
mdroidian May 14, 2024
6a356e5
check for comment before sending apiPost
mdroidian May 14, 2024
6db4cbf
re-add NODETEXT and NODEUID for sidebar support
mdroidian May 14, 2024
a2ffa74
handle sidebar pages
mdroidian May 14, 2024
8218df8
move props to sub component, store in localStorage (#255)
mdroidian May 14, 2024
57b2b2f
stop access-token checks if we've alread received from auth window
mdroidian May 14, 2024
d85d3f6
update docs
mdroidian May 14, 2024
72ed156
remove isVisible, replace with JSX conditional
mdroidian May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ For more information, check out our docs at [https://github.com/RoamJS/query-bui
- [Examples](https://github.com/RoamJS/query-builder/blob/main/docs/query-builder.md#examples)
- [Discourse Graphs](https://github.com/RoamJS/query-builder/blob/main/docs/discourse-graphs.md)
- [Native Roam Queries](https://github.com/RoamJS/query-builder/blob/main/docs/roam-queries.md#native-roam-queries)
- [GitHub Sync](https://github.com/RoamJS/query-builder/blob/main/docs/github-sync.md)
- [Creating Native Roam Queries](https://github.com/RoamJS/query-builder/blob/main/docs/roam-queries.md#creating-native-roam-queries)
- [Manipulating Native Roam Queries](https://github.com/RoamJS/query-builder/blob/main/docs/roam-queries.md#manipulating-native-roam-queries)
- [Sorting](https://github.com/RoamJS/query-builder/blob/main/docs/roam-queries.md#sorting)
Expand Down
56 changes: 56 additions & 0 deletions docs/github-sync.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# GitHub Sync

This extension implemnents GitHub Sync, allowing you to synchronize specific pages directly from Roam to a specified GitHub repository as Issues.

## Config Page

This will be found at `[[roam/js/github-sync]]` in your graph.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First thing that stood out to me in the video - thoughts on this living within the existing node config pages instead of introducing a new page? Couple of motivations behind this:

  • Reducing the amount of config locations the user needs to traverse too
  • If we keep things defined at the node level, it will make it easier to one day port as it will be a property of a SamePage native data model (the Discourse Node, or Node) rather than something specific to Roam <> Github

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially this felt far enough away from Discourse Graph to be on it's own. Then I ended up hooking into Discourse Nodes for their definition 😅. But I could see users want to use only this feature and none of the other dgraph features ... then it would be kind of buried / confusing.


### Node Select

Select which type of pages you want to sync to GitHub as Issues.

The list is made from defined [Discourse Graph Nodes](discourse-graphs.md).

### Comments Block

This is where the comments will live. An `Add Comment` button will appear on this block as well as a download button.

After you add a comment, that new comment block will have a `Add to GitHub` button to send it to the issue.

Clicking the download button will grab any new comments and open a dialog to confirm adding them to the block.

**Query Block Definition**

Define which block will be the Comments Block.

You can add the variables `:in NODETEXT` or `:in NODETITLE` which will grab the current pages's text or title.

Example:

![](media/github-sync-comment-query.png)

## Issue Page

### Send To GitHub

When you first navigate to a defined Issue Page, you will see two buttons under the title

![](media/github-sync-issue-page-title-1.png)

Click the Send To GitHub button to start the upload process. The process is as follows:

- install the SamePage GitHub App
- authorize the app to access your GitHub repository
Comment on lines +43 to +44
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's clarify below that these steps only need to happen once

- select the repository you want to send the issue to
- click `Export`

You will only need to do this once for each page you would like to sync.

Once this is complete, the title will just show the `GitHub Sync Details` Button. Clicking this button will show additional details about the issue, include a link to the issue, a link to settings, as well as the ability to re-authorize if required.

### Comments

Click the `Add Comment` button to add a comment to the page. Once the comment is created you should see a `Add to GitHub` button. Clicking this will add the comment to the issue. After it is sent, the `Add to GitHub` button should change to a `link` icon which will open the comment on GitHub.

Click the `Download Comments` button to fetch any new comments from the issue. You will see a confirmation dialog of all the comments that will be added.
Binary file added docs/media/github-sync-comment-query.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/github-sync-issue-page-title-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
157 changes: 126 additions & 31 deletions src/components/Export.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ import { getNodeEnv } from "roamjs-components/util/env";
import apiGet from "roamjs-components/util/apiGet";
import apiPut from "roamjs-components/util/apiPut";
import localStorageGet from "roamjs-components/util/localStorageGet";
import { ExportGithub } from "./ExportGithub";
import { ExportGithub, GitHubDestination } from "./ExportGithub";
import localStorageSet from "roamjs-components/util/localStorageSet";

const ExportProgress = ({ id }: { id: string }) => {
Expand Down Expand Up @@ -79,24 +79,27 @@ const ExportProgress = ({ id }: { id: string }) => {
);
};

const EXPORT_DESTINATIONS = [
{ id: "local", label: "Download Locally", active: true },
{ id: "app", label: "Store in Roam", active: false },
{ id: "samepage", label: "Store with SamePage", active: false },
{ id: "github", label: "Send to GitHub", active: true },
];

export type ExportDialogProps = {
results?: Result[] | ((isSamePageEnabled: boolean) => Promise<Result[]>);
title?: string;
columns?: Column[];
isExportDiscourseGraph?: boolean;
initialPanel?: "sendTo" | "export";
initialExportDestination?: (typeof EXPORT_DESTINATIONS)[number]["id"];
onClose?: () => void;
};

type ExportDialogComponent = (
props: RoamOverlayProps<ExportDialogProps>
) => JSX.Element;

const EXPORT_DESTINATIONS = [
{ id: "local", label: "Download Locally", active: true },
{ id: "app", label: "Store in Roam", active: false },
{ id: "samepage", label: "Store with SamePage", active: false },
{ id: "github", label: "Send to GitHub", active: true },
];
const exportDestinationById = Object.fromEntries(
EXPORT_DESTINATIONS.map((ed) => [ed.id, ed])
);
Expand All @@ -109,10 +112,8 @@ const ExportDialog: ExportDialogComponent = ({
title = "Share Data",
isExportDiscourseGraph = false,
initialPanel,
initialExportDestination,
}) => {
const [selectedRepo, setSelectedRepo] = useState(
localStorageGet("selected-repo")
);
const exportId = useMemo(() => nanoid(), []);
useEffect(() => {
setDialogOpen(isOpen);
Expand All @@ -139,7 +140,11 @@ const ExportDialog: ExportDialogComponent = ({
exportTypes[0].name
);
const [activeExportDestination, setActiveExportDestination] =
useState<string>(EXPORT_DESTINATIONS[0].id);
useState<string>(
initialExportDestination
? exportDestinationById[initialExportDestination].id
: EXPORT_DESTINATIONS[0].id
);
const [isSamePageEnabled, setIsSamePageEnabled] = useState(false);

const checkForCanvasPage = (title: string) => {
Expand All @@ -166,9 +171,6 @@ const ExportDialog: ExportDialogComponent = ({
const [includeDiscourseContext, setIncludeDiscourseContext] = useState(
discourseGraphEnabled as boolean
);
const [gitHubAccessToken, setGitHubAccessToken] = useState<string | null>(
localStorageGet("oauth-github")
);
const [canSendToGitHub, setCanSendToGitHub] = useState(false);

const writeFileToRepo = async ({
Expand All @@ -180,9 +182,15 @@ const ExportDialog: ExportDialogComponent = ({
content: string;
setError: (error: string) => void;
}): Promise<{ status: number }> => {
const base64Content = btoa(content);
const gitHubAccessToken = localStorageGet("github-oauth");
const selectedRepo = localStorageGet("github-repo");

const encoder = new TextEncoder();
const uint8Array = encoder.encode(content);
const base64Content = btoa(String.fromCharCode(...uint8Array));

try {
// https://docs.github.com/en/rest/repos/contents?apiVersion=2022-11-28#create-or-update-file-contents
const response = await apiPut({
domain: "https://api.github.com",
path: `repos/${selectedRepo}/contents/${filename}`,
Expand All @@ -195,7 +203,6 @@ const ExportDialog: ExportDialogComponent = ({
},
});
if (response.status === 401) {
setGitHubAccessToken(null);
setError("Authentication failed. Please log in again.");
localStorageSet("oauth-github", "");
return { status: 401 };
Expand All @@ -211,6 +218,73 @@ const ExportDialog: ExportDialogComponent = ({
return { status: 500 };
}
};
const writeFileToIssue = async ({
title,
body,
setError,
pageUid,
}: {
title: string;
body: string;
setError: (error: string) => void;
pageUid: string;
}): Promise<{ status: number }> => {
const gitHubAccessToken = localStorageGet("github-oauth");
const selectedRepo = localStorageGet("github-repo");
try {
// https://docs.github.com/en/rest/issues/issues?apiVersion=2022-11-28#create-an-issue
const response = await apiPost({
domain: "https://api.github.com",
path: `repos/${selectedRepo}/issues`,
headers: {
Authorization: `token ${gitHubAccessToken}`,
},
data: {
title,
body,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[oos] There will be formatting differences, like Roam bolding -> Github bolding.

This is where proxying through SamePage I think could be helpful, since we are defining the AtJson standard there to be interoperable across all text editors

// milestone,
// labels,
// assignees
},
});
if (response.status === 401) {
setError("Authentication failed. Please log in again.");
localStorageSet("oauth-github", "");
return { status: 401 };
}

if (response.status === 201) {
const props = getBlockProps(pageUid);
const newProps = {
...props,
["github-sync"]: {
issue: {
id: response.id,
number: response.number,
html_url: response.html_url,
state: response.state,
labels: response.labels,
createdAt: response.created_at,
updatedAt: response.updated_at,
repo: selectedRepo,
},
},
};
window.roamAlphaAPI.updateBlock({
block: {
uid: pageUid,
props: newProps,
},
});
}

return { status: response.status };
} catch (error) {
const e = error as Error;
setError("Failed to create issue");
return { status: 500 };
}
};

const handleSetSelectedPage = (title: string) => {
setSelectedPageTitle(title);
Expand Down Expand Up @@ -493,15 +567,12 @@ const ExportDialog: ExportDialogComponent = ({
onItemSelect={(et) => setActiveExportDestination(et)}
/>
</Label>
<ExportGithub
isVisible={activeExportDestination === "github"}
selectedRepo={selectedRepo}
setSelectedRepo={setSelectedRepo}
setError={setError}
gitHubAccessToken={gitHubAccessToken}
setGitHubAccessToken={setGitHubAccessToken}
setCanSendToGitHub={setCanSendToGitHub}
/>
{activeExportDestination === "github" && (
<ExportGithub
setError={setError}
setCanSendToGitHub={setCanSendToGitHub}
/>
)}
</div>
</div>

Expand Down Expand Up @@ -622,17 +693,41 @@ const ExportDialog: ExportDialogComponent = ({

if (activeExportDestination === "github") {
const { title, content } = files[0];
const githubDestination =
localStorageGet("github-destination");
try {
const { status } = await writeFileToRepo({
filename: title,
content,
setError,
});
let status;
if (githubDestination === "File") {
status = (
await writeFileToRepo({
filename: title,
content,
setError,
})
).status;
}
if (githubDestination === "Issue") {
const pageUid =
typeof results === "function" ? "" : results[0].uid; // TODO handle multiple results
if (!pageUid) {
setError("No page UID found.");
return;
}
status = (
await writeFileToIssue({
title: title.replace(/\.[^/.]+$/, ""), // remove extension
body: content,
setError,
pageUid,
})
).status;
}

if (status === 201) {
// TODO: remove toast by prolonging ExportProgress
renderToast({
id: "export-success",
content: "Upload Success",
content: `Upload Success to ${githubDestination}`,
intent: "success",
});
onClose();
Expand Down
Loading