Ingest Calamari Samples #631

lesliech1004 · 2025-06-23T18:55:04Z

Ingest Emily Calamari's samples of UCDs with higher mass primaries. Sources, publications, companion relationships.

Link to relevant issue: Closes #633
Closes #636

For data ingests:

includes script used for ingest
includes modified JSON files
Add new tests
Update the Versions table

I need the ra and dec data for the primary

…tionships code

…004/SIMPLE-db into ingest_calamari_sample

data/source/ε_indi_a.json

scripts/ingests/calamari/ingest_calamari_samples.py

kelle · 2025-07-02T16:03:35Z

Looks super great! I need to sit down and give it a more careful review but script and JSONs look excellent.

kelle

Some small comments, one big one: primary stars do not need to be ingested.

kelle · 2025-07-07T18:19:50Z

data/reference/Publications.json

+        "description": "SCExAO/CHARIS Direct Imaging Discovery of a 20 au Separation, Low-mass Ratio Brown Dwarf Companion to an Accelerating Sun-like Star"
+    },
+    {
+        "reference": "Gaia21",


already ingested as GaiaEDR3

kelle · 2025-07-07T18:27:17Z

data/source/_47_oph.json

+{
+    "Sources": [
+        {
+            "source": "* 47 Oph",


try ingesting source without "*"

kelle · 2025-07-07T18:27:29Z

data/source/_eta_crb_ab.json

+{
+    "Sources": [
+        {
+            "source": "* eta CrB AB",


try ingesting source without "*"

data/source/eps_indi_b.json

kelle · 2025-07-07T18:28:54Z

data/source/ε_indi_a.json

Pretty sure this file should be deleted.

kelle · 2025-07-07T18:33:40Z

scripts/ingests/calamari/ingest_calamari_samples.py

+logger.info(f"references ingested:{ref_ingested}")  # 10 references ingested
+logger.info(f"references already exists:{ref_already_exists}")  # 24 references due to preexisting data
+logger.info(f"total references:{ref_ingested+ref_already_exists}")  # 34 references total
+logger.info(f"sources ingested:{sources_ingested}")  # 42 ingested
+logger.info(f"sources already exists:{sources_already_exists}")  # 74 due to preexisting data
+logger.info(f"total sources:{sources_ingested+sources_already_exists}")  # 116 sources total
+logger.info(f"companion relationships ingested:{companions_ingested}")  # 101 ingested
+logger.info(f"companion relationships already exists:{companions_already_exists}")  # 15 due to preexisting data
+logger.info(f"total companion relationships:{companions_ingested+companions_already_exists}")  # 116 total


Do all of these make sense given the needed changes to the tests?

kelle · 2025-07-07T18:38:25Z

data/source/hd_33632.json

We don't need this source ingested, just it's child.

kelle · 2025-07-07T18:42:52Z

data/source/hd_253662b.json

            "other_name": "2MASS J06135342+1514062"
        },
+        {
+            "other_name": "HD 253662"


This in the parent, not another name for the source.

kelle · 2025-07-07T18:54:14Z

scripts/ingests/calamari/ingest_calamari_samples.py

please move the scripts related to ingesting publications to their own script. We also want to think about how to best add ingesting from a URL capability to ingest_publication.

Co-authored-by: Kelle Cruz <kellecruz@gmail.com>

…004/SIMPLE-db into ingest_calamari_sample

lesliech1004 · 2025-07-09T15:33:24Z

When ingesting the primaries as the unresolved parent, it returns an error saying that the source doesn't exist. This is because I removed the ingest code for the primaries. Should I just not ingest these unresolved parents or should I write the code to ingest the primaries? I predict this issue will also pop up when ingesting the primaries as parents of the objects. Should I not ingest these relationships?

kelle · 2025-07-09T15:49:06Z

When ingesting the primaries as the unresolved parent, it returns an error saying that the source doesn't exist. This is because I removed the ingest code for the primaries. Should I just not ingest these unresolved parents or should I write the code to ingest the primaries? I predict this issue will also pop up when ingesting the primaries as parents of the objects. Should I not ingest these relationships?

I think all of the unresolved parents need to also be ingested as sources. This does not mean ingest all primaries, just the ones that are unresolved parents. Might be easiest to do this "by hand" or source-by-source.

Also, I noticed that the "ingest_companion_relationships" function (simple/utils/companions.py) will need some modification to allow the new "unresolved child" relationship.

lesliech1004 · 2025-07-09T16:13:15Z

When ingesting the primaries as the unresolved parent, it returns an error saying that the source doesn't exist. This is because I removed the ingest code for the primaries. Should I just not ingest these unresolved parents or should I write the code to ingest the primaries? I predict this issue will also pop up when ingesting the primaries as parents of the objects. Should I not ingest these relationships?

I think all of the unresolved parents need to also be ingested as sources. This does not mean ingest all primaries, just the ones that are unresolved parents. Might be easiest to do this "by hand" or source-by-source.

Also, I noticed that the "ingest_companion_relationships" function (simple/utils/companions.py) will need some modification to allow the new "unresolved child" relationship.

Wait, I also think the issue is that some of the primary names are not SIMBAD resolvable. After searching the database, the function searches SIMBAD for the source's name. IF the name is unresolvable in SIMBAD, this returns no source found.

kelle · 2025-07-09T16:48:45Z

Aha! Good find.

modify the ingest_companion_relationship function to give a better error message
search Simbad and see if there's a better name

This reverts commit 1bfdb37.

lesliech1004 and others added 12 commits June 23, 2025 14:44

demo code for ingesting publication for the calamari samples

44eb869

Updated

3f27c31

Updated and finished ingest publications script for calamari samples

e6d648f

Ingest calamari sample sources and other references

f40e3a6

I need the ra and dec data for the primary

Calamari samples ingest source code. Unfinished ingest companion rela…

f0f78fb

…tionships code

Ingested the sources, names, and companion relationships

6da94d2

Comments

7014d85

Merge branch 'main' into ingest_calamari_sample

8f5811f

Updated pytest cases and JSON files

1bfdb37

Merge branch 'ingest_calamari_sample' of https://github.com/lesliech1…

dbfcb3c

…004/SIMPLE-db into ingest_calamari_sample

Updated test cases

ddde91f

test case update

2a39dd9

lesliech1004 requested a review from kelle July 2, 2025 15:30

kelle reviewed Jul 2, 2025

View reviewed changes

data/source/ε_indi_a.json Outdated Show resolved Hide resolved

scripts/ingests/calamari/ingest_calamari_samples.py Outdated Show resolved Hide resolved

lesliech1004 added 6 commits July 2, 2025 13:19

Delete data/source/eps_indi_a.json

0ffc663

Update test_data.py

a7f47bb

Update test_data.py

c532431

Update test_data.py

2905538

Merge branch 'main' into ingest_calamari_sample

06adaf1

Update test_data.py

514cc55

kelle requested changes Jul 7, 2025

View reviewed changes

kelle reviewed Jul 7, 2025

View reviewed changes

lesliech1004 and others added 3 commits July 9, 2025 11:05

Update data/source/eps_indi_b.json

3fc50ba

Co-authored-by: Kelle Cruz <kellecruz@gmail.com>

Updated ingest codes. Separated into different files

d4aa926

Merge branch 'ingest_calamari_sample' of https://github.com/lesliech1…

32d4efa

…004/SIMPLE-db into ingest_calamari_sample

kelle mentioned this pull request Jul 9, 2025

modify and separate JSON files for brown dwarf multiples. #641

Closed

lesliech1004 added 5 commits July 11, 2025 15:54

Updated code. Ingesting the unresolved parents and resolved children.

ae34032

undo push

ea1f9b1

Revert "Updated pytest cases and JSON files"

d696a93

This reverts commit 1bfdb37.

Revert "Updated pytest cases and JSON files"

80a1e47

This reverts commit 1bfdb37.

Revert "Updated pytest cases and JSON files"

d786b28

This reverts commit 1bfdb37.

lesliech1004 closed this Jul 14, 2025

Ingest Calamari Samples #631

Ingest Calamari Samples #631

Uh oh!

Conversation

lesliech1004 commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kelle commented Jul 2, 2025

Uh oh!

kelle left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lesliech1004 commented Jul 9, 2025

Uh oh!

kelle commented Jul 9, 2025

Uh oh!

lesliech1004 commented Jul 9, 2025

Uh oh!

kelle commented Jul 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lesliech1004 commented Jun 23, 2025 •

edited

Loading