Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
name: Bug report
about: Create a report to help us improve
title: "[Bug]"
labels: ''
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Provide an example url to reproduce the behavior.

\*Reminder: If authentication is required **do not** leave any sensitive credentials in the snippet.


**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: [e.g. Ubuntu 20.04]
- Python Version [e.g. python3.11]
- Pip Environment ['python3 -m pip freeze']

**Additional context**
Add any other context about the problem here.
43 changes: 43 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Merge Requirements:
The following requirements must be met for your pull request to be considered for review & merging. Until these requirements are met please mark the pull request as a draft.

## Purpose
Why is this pull request necessary? Provide a reference to a related issue in this repository that your pull request addresses (if applicable).

## Description
A brief description of the changes proposed in the pull request. If there are any changes to packaging requirements please list them. If it's a new endpoint list:
- Supported HTTP methods
- input params
- output
- errors it can raise

## Snippet
If the pull request provides a new feature, provide an example demonstrating the use-case(s) for this pull request (If applicable).

For example, if you are adding a new endpoint, show how we might call it and what kind of output we could expect:
``` bash
curl 'http://127.0.0.1:8080/services/utils/useful_new_endpoint?param1=value1&param2=value2'
```

If it modifies an existing endpoint (like a new output type for `/services/search/param`) show and example of what the output would look like.

## Error/Warning/Regression Free
Your code runs without any unhandled errors, warnings, or regressions

## Unit Tests

You have added unit tests to the test suite see the [README Testing section](https://github.com/asfadmin/Discovery-SearchAPI-v3/tree/dev?tab=readme-ov-file#writing-tests) for an overview on adding tests to the test suite.

## Target Merge Branch
Your pull request targets the `dev` branch


***

### Checklist
- [ ] Purpose
- [ ] Description
- [ ] Snippet
- [ ] Error/Warning/Regression Free
- [ ] Unit Tests
- [ ] Target Merge Branch
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,13 @@ and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
------
## [1.0.8](https://github.com/asfadmin/Discovery-SearchAPI-v3/compare/v1.0.7...v1.0.8)
### Changed
- bump asf-search to v10.0.2 for NISAR product type file sizes, urgent response now searchable with product types, and ARIA-S1 GUNW Stacking support
- bump asf-search to v10.0.4 for NISAR product type file sizes, urgent response now searchable with product types, and ARIA-S1 GUNW Stacking support

### Fixed
- boolean values are properly capitalized in `python` output file
- API maturity set for each level of deployment stage
- API maturity loaded once per api instance

------
## [1.0.7](https://github.com/asfadmin/Discovery-SearchAPI-v3/compare/v1.0.6...v1.0.7)
### Changed
Expand Down
154 changes: 147 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,149 @@
# SearchAPI-v3

- Login to aws console using Kion
- Find the region that has the VPC and note account number, vpc_id, subnet_ids and security_group. Subnet_ids should be a comma separated list
- Get temp cli credentials from Kion and add them to your shell
- Run CDK bootstrap in region with the VPC using cdk-bootstrap-example.sh and filling in the account number, vpc_id, subnet_ids and security_group
- If not already created, make an GitHubActionsOidcProvider using the cdk/oidc/oidc-provider.yml template
- Create OIDC role using cloudformation template cdk/oidc/github-actions-oidc.yml. For 'ActionsRoleName' parameter put 'SearchAPIActionsOIDCRole'
- Create a github environment with params using the same values from the CDK bootstrap. AWS_ACCOUNT_ID, SECURITY_GROUP, SUBNET_IDS, VPC_ID
SearchAPI-v3 is a wrapper around the [asf-search python module](https://github.com/asfadmin/Discovery-asf_search) using a serverless deployment with the FastAPI web framework and AWS lambda.

### Main Endpoints

<table>
<thead>
<tr>
<th>Endpoint</th>
<th>Descirption</th>
<th>Methods</th>
</tr>
</thead>
<tbody>
<tr>
<td>`/`</td>
<td>server configuration info</td>
<td>`GET`</td>
</tr>
<tr>
<td>`/health`</td>
<td>same as root `/`</td>
<td>`GET`</td>
</tr>
<tr>
<td>`/services/search/param`</td>
<td>Search via any valid asf-search parameters</td>
<td>`GET` `POST` `HEAD`</td>
</tr>
<tr>
<td>`/services/search/baseline`</td>
<td>Create a baseline stack based off a given reference and optional dataset</td>
<td>`GET` `POST` `HEAD`</td>
</tr>
</tbody>
</table>

## Development

### Branching

<table>
<thead>
<tr>
<th>Instance</th>
<th>Branch</th>
<th>Description, Instructions, Notes</th>
</tr>
</thead>
<tbody>
<tr>
<td>Features</td>
<td>feat-*</td>
<td>Always branch off dev, for new features</td>
</tr>
<tr>
<td>Issues</td>
<td>bugfix-*</td>
<td>Always branch off dev, for bugfixes</td>
</tr>
<tr>
<td>development</td>
<td>dev</td>
<td>Beginning integration testing happens here, deploys to test-staging deployment</td>
</tr>
<tr>
<td>testing</td>
<td>test</td>
<td>Only accepts merges from development, test deployment</td>
</tr>
<tr>
<td>production staging and integration testing</td>
<td>prod-staging</td>
<td>Only acepts merges from testing, deploys to prod-staging deployment</td>
</tr>
<tr>
<td>release</td>
<td>prod</td>
<td>only accepts merges from prod-staging, release branch, prod deployment</td>
</tr>
</tbody>
</table>

### Installation

To install locally run the following in a terminal (we highly recommend installing using a virtual environment)
```bash
pip install -r requirements.txt
pip install .
```

To install test requirements, run
```bash
pip install -r tests/requirements.txt
```

### Running Locally

To run the API locally run the following in a terminal
```bash
uvicorn src.SearchAPI.application:app --reload --port 8080
```
The api should now be available via your localhost at http://127.0.0.1:8080 and can be opened and queried with your browser or network tool of choice.




## Testing

## Running the Test Suite Locally
After running the API (see `Running Locally` above), in order to run the test suite locally run the following:
```bash
pytest --api "http://127.0.0.1:8080" -n auto "tests/yml_tests/"
```

## Writing tests
Tests should be written to relevant subfolder & files in `/tests`

The test suite uses the `pytest-automation` pytest plugin which allows us to define and re-use input for test cases in the yaml format. Test cases are written to files in `tests/yml_tests/`, and reusable resources for those tests `tests/yml_tests/Resources/`.

```yaml

tests:
- Test Nisar Product L1 RSLC: # this is a test case
product: NISAR_L1_PR_RSLC_087_039_D_114_2005_DHDH_A_20251102T222008_20251102T222017_T00407_N_P_J_001.yml # this file should be in `tests/yml_tests/Resources/`. See other yml files in the folder to see how you might structure the yml object
product_level: L1

- Test Nisar Product L2 GSLC: # this is another test case
product: NISAR_L2_PR_GSLC_087_039_D_112_2005_DHDH_A_20251102T221859_20251102T221935_T00407_N_F_J_001.yml
product_level: L2
```

We can create the mapping from our yaml test cases in `tests/yml_tests/pytest-config.yml`, which will be used to call the desired python function in `tests/yml_tests/pytest-managers.py`

In `tests/yml_tests/pytest-config.yml`:
```yaml
- For running ASFProduct tests:
required_keys: ['product', 'product_level'] # the keys the test case requires
method: test_NISARProduct # the python function in pytest-managers.py that will be called
required_in_title: Test Nisar Product # (OPTIONAL) will only run test cases defined with `Test Nisar Product` in the name, so the above two test cases would be run with our tests.
```


In `tests/yml_tests/pytest-managers.py`:
```python
def test_new_endpoint(**args) -> None: # Must match the name in pytest-config.yml like above for `method`
test_new_endpoint(client=client, **args)
```
18 changes: 8 additions & 10 deletions src/SearchAPI/application/application.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@
allow_headers=["*"],
)

cfg = load_config_maturity()
cmr_health = get_cmr_health(cfg['cmr_base'], cfg['cmr_health'])


@router.api_route("/services/search/param", methods=["GET", "POST", "HEAD"])
async def query_params(searchOptions: SearchOptsModel = Depends(process_search_request)):
Expand Down Expand Up @@ -226,18 +229,16 @@ async def file_to_wkt(files: list[UploadFile]):
headers=constants.DEFAULT_HEADERS
)

# @router.get('/redirect/{short_name}/{granule_id}')
# async def nisar_static_layer(environment: Literal['prod', 'test'], short_name: str, granule_id: str):
# example: https://api.daac.asf.alaska.edu/services/redirect/NISAR_L2_STATIC/{granule_id}.h5
# @router.get('/services/redirect/{short_name}/{granule_id}')
# async def nisar_static_layer(short_name: str, granule_id: str):
# """
# environment: 'prod' or 'test' (whether to search the cmr prod or uat record)
# short_name: the CMR static layer collection short name to search
# granule_id: the granule id of the product to find the static layer for

# returns: redirect to file url
# """
# opts = asf.ASFSearchOptions()
# if environment == 'test':
# opts.host = asf.INTERNAL.CMR_HOST_UAT
# opts = asf.ASFSearchOptions(host=cfg['cmr_base'])

# try:
# granule = asf.search(
Expand Down Expand Up @@ -281,14 +282,11 @@ async def health_check():
api_logger.info(exc)
api_version = {'version': 'unknown'}

cfg = load_config_maturity()
cmr_health = get_cmr_health(cfg['cmr_base'], cfg['cmr_health'])

api_health = {
'ASFSearchAPI': {
'ok?': True,
'version': api_version['version'],
'config': load_config_maturity()
'config': cfg
},
'CMRSearchAPI': cmr_health
}
Expand Down