-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathwhy-podpac.html
More file actions
199 lines (180 loc) · 11 KB
/
why-podpac.html
File metadata and controls
199 lines (180 loc) · 11 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Why PODPAC? — 3.2.0+3.gb2780d4a</title>
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/style.css" type="text/css" />
<link rel="shortcut icon" href="_static/favicon.png"/>
<link rel="canonical" href="https://podpac.org/why-podpac.html" />
<!--[if lt IE 9]>
<script src="_static/js/html5shiv.min.js"></script>
<![endif]-->
<script src="_static/jquery.js"></script>
<script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/sphinx_highlight.js"></script>
<script src="_static/js/theme.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Design" href="design.html" />
<link rel="prev" title="AWS Integration" href="aws.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="index.html">
<img src="_static/icon.svg" class="logo" alt="Logo"/>
</a>
<div class="version">
3.2.0
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<p class="caption" role="heading"><span class="caption-text">Getting Started</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="overview.html">Overview</a></li>
<li class="toctree-l1"><a class="reference internal" href="install.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="settings.html">Settings</a></li>
<li class="toctree-l1"><a class="reference internal" href="examples.html">Examples</a></li>
<li class="toctree-l1"><a class="reference internal" href="aws.html">AWS Integration</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Topics</span></p>
<ul class="current">
<li class="toctree-l1 current"><a class="current reference internal" href="#">Why PODPAC?</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#problems">Problems</a></li>
<li class="toctree-l2"><a class="reference internal" href="#podpac">PODPAC</a></li>
<li class="toctree-l2"><a class="reference internal" href="#other-related-projects">Other related projects</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="design.html">Design</a></li>
<li class="toctree-l1"><a class="reference internal" href="dependencies.html">Dependencies</a></li>
<li class="toctree-l1"><a class="reference internal" href="nodes.html">Nodes</a></li>
<li class="toctree-l1"><a class="reference internal" href="coordinates.html">Coordinates</a></li>
<li class="toctree-l1"><a class="reference internal" href="cache.html">Cache</a></li>
<li class="toctree-l1"><a class="reference internal" href="datasets.html">Supported Datasets</a></li>
<li class="toctree-l1"><a class="reference internal" href="interpolation.html">Interpolation</a></li>
<li class="toctree-l1"><a class="reference internal" href="earthdata.html">NASA Earth Data Login</a></li>
<li class="toctree-l1"><a class="reference internal" href="aws-development.html">AWS Development</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">References</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="api.html">API Reference</a></li>
<li class="toctree-l1"><a class="reference internal" href="wrapping-datasets.html">Wrapping Datasets</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Support</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="references.html">References</a></li>
<li class="toctree-l1"><a class="reference internal" href="contributing.html">Contributing</a></li>
<li class="toctree-l1"><a class="reference internal" href="docs.html">Documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="roadmap.html">Development Roadmap</a></li>
<li class="toctree-l1"><a class="reference internal" href="changelog.html">Changelog</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">PODPAC</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item active">Why PODPAC?</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/why-podpac.md.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="why-podpac">
<h1>Why PODPAC?<a class="headerlink" href="#why-podpac" title="Permalink to this heading"></a></h1>
<section id="problems">
<h2>Problems<a class="headerlink" href="#problems" title="Permalink to this heading"></a></h2>
<p>Observational and modeled data products from agencies such as NASA, ESA, and academic institutions encompass terabytes
to petabytes of scientific data available for analysis, analytics, and exploitation. Unfortunately, these data sets are
highly underutilized by the scientific community due to:</p>
<ul class="simple">
<li><p>Vast computational resource requirements</p></li>
<li><p>Disparate formats, projections, and resolutions that hinder data fusion and integrated analyses across different data sets</p></li>
<li><p>Complex and disjoint data access and retrieval protocols</p></li>
<li><p>Task specific and non-reusable code development processes that hinder algorithm sharing and collaboration</p></li>
</ul>
<p>Moreover, NASA programs such as Earth Observing System Data and Information System (EOSDIS) are actively investigating
migration of their vast data archives to storage on commercial cloud services such as Amazon Web Services (AWS). In
order to maximize the benefit of cloud-based data storage, it is necessary to also enable capabilities for cloud-based
data analysis and analytics so that data processing occurs “close” to where it is stored and also exploits the powerful
resources of highly distributed cloud services. However, the ability to deploy scalable, distributed cloud-based data
analyses and analytics requires a high degree of cloud computing expertise, and thus greatly exceeds the current
capabilities of typical NASA and non-NASA earth scientists.</p>
</section>
<section id="podpac">
<h2>PODPAC<a class="headerlink" href="#podpac" title="Permalink to this heading"></a></h2>
<p>PODPAC is an open-source, standards-based Python software framework that removes major barriers to
widespread exploitation of earth science data and cloud-based distributed data processing. PODPAC uses a pipeline-based software architecture that</p>
<ol class="arabic simple">
<li><p>Enables multiscale and multi-windowed access, exploration, and integration of available earth science data sets to
support both analysis and analytics</p></li>
<li><p>Automatically accounts for differences in underlying geospatial data formats, projections, and resolutions</p></li>
<li><p>Greatly simplifies the implementation and parallelization of geospatial data processing routines</p></li>
<li><p>Directly integrates with advanced machine learning frameworks and other open source Python libraries</p></li>
<li><p>Unifies the access, processing, and sharing of data and algorithms through easy-to-use interfaces to existing
NASA data repositories</p></li>
<li><p>Enables scientists with minimal expertise in cloud computing to seamlessly transition data product pipelines
developed on a local workstation to execution via scalable, highly distributed cloud processing</p></li>
</ol>
</section>
<section id="other-related-projects">
<h2>Other related projects<a class="headerlink" href="#other-related-projects" title="Permalink to this heading"></a></h2>
<p>There are other excellent open source Python projects that address aspects of these problems. In fact, PODPAC uses many of these, and aims to incorporate a multitude of useful tools within our approach for dealing with geospatial data.</p>
<ul class="simple">
<li><p><a class="reference external" href="http://xarray.pydata.org/en/stable/index.html">xarray</a></p></li>
<li><p><a class="reference external" href="http://geopandas.org/">Geopandas</a></p></li>
<li><p><a class="reference external" href="https://dask.pydata.org/en/latest/">Dask</a></p></li>
<li><p><a class="reference external" href="https://unidata.github.io/MetPy/latest/index.html">metpy</a></p></li>
<li><p><a class="reference external" href="https://rasterio.readthedocs.io/en/latest/">rasterio</a></p></li>
<li><p><a class="reference external" href="http://pangeo-data.org/">Pangeo</a></p></li>
<li><p><a class="reference external" href="https://www.opendatacube.org/">OpenDataCube</a></p></li>
</ul>
</section>
</section>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="aws.html" class="btn btn-neutral float-left" title="AWS Integration" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="design.html" class="btn btn-neutral float-right" title="Design" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright 2017-2023, Creare.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>