Merge pull request #31 from saberbrasher/content-mapping

frizatch · web-flow · commit 16dc8520c534 · 2025-10-02T17:39:56.000-06:00
Content mapping version 2 (Saber's edition)
diff --git a/.gitignore b/.gitignore
@@ -4,3 +4,4 @@ example_data/
 *.h5
 *.nc
 _freeze
+.venv/
diff --git a/_quarto.yml b/_quarto.yml
@@ -27,8 +27,6 @@ website:
         href: index.qmd
       - section: "Getting Started"
         contents:
-          - text: "How to use this cookbook"
-            href: our-cookbook.qmd
           - text: "Getting an Earthdata Login"
             href: reference-guides/get-edl.qmd            
           - text: "Tools we use"
@@ -38,48 +36,46 @@ website:
       - section: "NSIDC Data"
         # href: reference-guides/data-orient-overview.qmd
         contents:
-           - text: "Types of data"
+           - text: "Data types and formats"
              href: reference-guides/data-structures.qmd
-           - text: "Common file formats"
-             href: reference-guides/nsidc-file-formats.qmd
            - text: "NASA collections and granules"
            - text: "NSIDC map projections and grids"
              href: reference-guides/projections.qmd
-           - text: "Quirky Datasets"
+           - text: "Quirky datasets"
              href: reference-guides/datasets.qmd
       - section: "Working with Data"
         # href: reference-guides/working-with-data-overview.qmd
         contents:
-          - section: "Finding and Accessing Data"
+          - section: "Finding / accessing data"
             contents:
               - text: "Data search and discovery"
                 href: reference-guides/dataset-search-and-discovery.qmd
               - text: "Data access"
                 href: reference-guides/data-access.qmd
-          - section: "Wrangling Data"
+          - section: "Wrangling data"
             contents:
               - text: "Subsetting"
               - text: "Reformatting"
               - text: "Reprojecting"
               - text: "Resampling"
-          - section: "Visualizing Data"
+          - section: "Visualizing data"
             contents:
               - text: "Plotting data - time series, scatter plots"
               - text: "Plotting data on a map"
                 href: reference-guides/plotting-data.qmd
-      - section: "How do I..."
+      - section: "How-To Guides"
         href: how-to-guides/overview.qmd
         contents:
           - section: "Search for data"
           - section: "Access data"
           - section: "Reproject data"
-          - section: "Work with NetCDF files?"
+          - section: "Work with NetCDF files"
             contents:
-            - text: "get the bounding box of a netcdf file"
+            - text: "Get the bounding box of a NetCDF file"
               href: how-to-guides/netcdf_cf.qmd
-            - text: "get the latitude and longitudes for grid cells"
+            - text: "Get the latitude and longitudes for grid cells"
               href: how-to-guides/get_latitude_and_longitude.qmd
-      - section: "Tutorials"
+      - section: "Tutorials / Workshops"
         href: tutorials/tutorials_overview.qmd
         contents:
           - text: "Direct cloud access of ICESat-2 data"
@@ -88,6 +84,7 @@ website:
             href: tutorials/working_with_icesat2_sea_ice_data.ipynb
       - section: "Appendices"
         contents:
+          - text: "Contributing"
           - text: "Best Practices"
           - text: "What is an API?"
 
diff --git a/how-to-guides/netcdf_cf.qmd b/how-to-guides/netcdf_cf.qmd
@@ -1,5 +1,5 @@
 ---
-title: "How do I get the bounding box of a NetCDF file in latitude and longitude?"
+title: "How to get the bounding box of a NetCDF file in latitude and longitude"
 author: Andrew P. Barrett
 date: last-modified
 ---
diff --git a/how-to-guides/overview.qmd b/how-to-guides/overview.qmd
@@ -1,27 +1,23 @@
 ---
 title: "How-To Guides"
-author: Andy P. Barrett
 ---
 
 ## Introduction
 
-This section of the cookbook contains **How To Guides** to help you
+This section of the cookbook contains **How-To Guides** to help you
 solve a particular problem or task.  You can think of these How-To
-Guides as recipes.  For the most part they are short.  They are
+Guides as programmatic recipes.  For the most part they are short.  They are
 written with the assumption that you know what you want to do and have
-some understanding of programing.  You can think of each How-To as a
+some understanding of programming.  You can think of each How-To as a
 building block that can be put together with other How-Tos to construct
 a workflow.
 
-If you are just learning either programming or working in the cloud,
-we recommend looking at the other chapters in this cookbook.
-
 How-To Guides are organized into the following sections:
-_Maybe add a short description of each section here_
 
-- How do I search for data; 
-- How do I accessing data;
-- How do I reproject and resample data;
-- How do I work with CF-compliant NetCDF files
-  - [How do I get the bounding box of a NetCDF file in latitude and longitude?](netcdf_cf.qmd)
+- How to search for data (via `earthaccess`); 
+- How to access data (via `earthaccess`);
+- How to reproject and resample data;
+- How to work with CF-compliant NetCDF files:
+  - [How to get the bounding box of a NetCDF file in latitude and longitude](netcdf_cf.qmd)
+  - [How to get latitude and longitude for the grid cells of a NetCDF file](get_latitude_and_longitude.qmd)
 
diff --git a/index.qmd b/index.qmd
@@ -3,14 +3,35 @@ title: "NSIDC Data Cookbook"
 author: NSIDC
 ---
 
-## Welcome
-
-Welcome to the NSIDC Data Cookbook!
-
 <!-- prettier-ignore -->
 > [!WARNING]
 > This cookbook is under active development. Major changes to the structure of
 > the book and its content are expected. We are striving to develop content
 > that is well-tested and peer-reviewed, but nothing contained here should be
 > expected to work correctly (or at all!) in this early phase.  Many sections contain only an outline of the content.
-> These sections will have content added at the project develops.
+> These sections will have content added as the project develops.
+
+## Welcome to the NSIDC Data Cookbook! 
+
+This Cookbook for National Snow and Ice Data Center Distributed Active Archive Center ([NSIDC DAAC](https://nsidc.org/data/data-programs/nsidc-daac)) Data is more Julia Child’s “Mastering the Art of French Cooking” than just a collection of recipes (or data tutorials). The aim is to not only provide easy to follow recipes for working with data but also to provide an understanding of the data managed by NSIDC and the tools available to work with the data. It is not an in-depth guide to the many datasets housed by NSIDC but instead is an introduction to the common types of data, file formats and data structures. The hope is that the cookbook will provide a guide and foundataion to help you *master the art of working with cryospheric data*.
+
+## How to use this cookbook
+
+If you are new to NASA or NSIDC data, begin with the **Getting Started** section. Beyond that, the cookbook is divided into four major sections: **NSIDC Data**, **Working with Data**, **How-To Guides**, and **Tutorials / Workshops**. We also include some **Appendices** that include a best practices guide for Python scripting, some background details on computing, and how to contribute to the cookbook (if you are so inclined!).
+
+The **NSIDC Data** section is an introduction to the types of data managed by NSIDC, the file formats used to store that data, coordinate reference systems and grids comon to NSIDC data, and a guide to NASA terminology for Earth science data. There is also an introduction to some “quirky” datasets that do not fit more common simple data structures or are just a little more difficult to work with.
+
+The **Working with Data** section is a guide to the tools and applications for accomplishing common steps in scientific workflows: finding and accessing data, wrangling data, and visualizing data. Wrangling data covers subsetting, reformatting, reprojecting and resampling data in preparation for analysis.
+
+The **How-To Guides** section is a list of recipes to acheive common tasks. It is intended a list of solutions to common scientific programming tasks.
+
+Lastly, the **Tutorials / Workshops** section serves to highlight content that was developed for past events or specific use cases.
+
+## About the NSIDC DAAC
+
+NSIDC has managed the NASA National Snow and Ice Data Center Distributed Active Archive Center (NSIDC DAAC) since 1993, archiving and distributing cryospheric and related geophysical data from NASA Earth-observing satellite missions, airborne campaigns and field observations. The NSIDC DAAC provides hundreds of free and open-access NASA Earth science data sets, detailed data documentation, data tools, resources and tutorials, as well as  robust data user support services. These data can be used to study topics relating to snow cover, sea ice, ice sheets, ice shelves, glaciers, frozen ground, soil moisture, climate interactions, and more!
+
+
+
+
+
diff --git a/reference-guides/data-structures.qmd b/reference-guides/data-structures.qmd
@@ -1,12 +1,23 @@
 ---
-title: "Data Structures"
+title: "Data types and formats"
 ---
 
-Describes common remote sensing data structures.
+There are many common data types/ structures and terminology to go along with them. Some examples:
 
-_We should add information for tabular data and Data Frames_
+- **Tabular** – rows and columns, often stored in CSV or TSV files. Each row is an observation, and each column is a variable (e.g., time, latitude, longitude, temperature).  
+- **Data Frames** – tabular data structures used in programming languages like R or Python (pandas). Data frames allow for more complex indexing, metadata, and transformations than simple tabular files.  
+- **Swath** – along-track measurements collected as the satellite passes over an area, usually irregular in shape and resolution.  
+- **Raster / Grids** – data organized into regular grid cells, each cell representing a spatial unit (e.g., 25 km × 25 km grid of snow cover).  
+- **Resampling** – methods for transforming data between swath, raster, or other structures (e.g., nearest neighbor, bilinear interpolation).  
 
-- Swath  
-- Along-track  
-- Raster/Grids  
-- Resampling
+
+How to work with file formats commonly found at NSIDC: In most cases, it’s best to avoid low-level libraries such as `netCDF4` or `h5py`. Higher-level libraries provide more intuitive access, automatically handle metadata, and streamline analysis. Some format descriptions and reccomendations  are in the table below.
+
+| File Format           | Description                                                                                 | Recommended Tools                                        |
+| --------------------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------- |
+| **NetCDF4 / NetCDFx** | Multidimensional climate/remote sensing data (time, lat, lon, variables).                   | `xarray` (`xr.open_dataset`) in Python; `terra` or `ncdf4` in R.    |
+| **HDF5**              | Hierarchical format for storing arrays, tables, and metadata; used widely in NASA products. | `xarray`, `pandas`; avoid `h5py` unless necessary.       |
+| **HDF-EOS**           | Earth Observing System variant of HDF, often with swath, grid, or point structures.         | `xarray`, `h5netcdf`, NASA `harmony-py`.                 |
+| **Shapefile**         | Vector geospatial data (points, lines, polygons) with CRS support.                          | `geopandas` (Python); `sf` (R).                          |
+| **GeoTIFF**           | Georeferenced raster imagery and gridded data.                                              | `rasterio`, `rioxarray` (Python); `terra`, `raster` (R). |
+| **CSV/TSV**           | Tabular text-based files, rows = observations, columns = variables.                         | `pandas` (Python); `readr`/`data.table`/`tibble` (R).    |
diff --git a/reference-guides/dataset-search-and-discovery.qmd b/reference-guides/dataset-search-and-discovery.qmd
@@ -26,7 +26,7 @@ The NSIDC Data Access Tool is accessible from landing pages on the NSIDC website
 
 #### **Programmatic**
 
-The majority of the shorter form "How Do I's" and longer form "Tutorials" will present programmatic data access methods using Python.  We hope to expand to other languages, such as R, in the future.  
+The majority of the shorter form "How-To Guides" and longer form "Tutorials / Workshops" will present programmatic data access methods using Python.  We hope to expand to other languages, such as R, in the future.  
 
 **earthaccess Python library**
 
diff --git a/tools-we-use.qmd b/tools-we-use.qmd
@@ -3,7 +3,7 @@ title: "Tools we use"
 #number-sections: true
 ---
 
-Currently, this cookbook features Python Packages (@sec-pythonpkg) for working with data supported by NSIDC DAAC.  It also features some applications (@sec-applications) that are accessible through a web browser or as stand alone packages that need to be installed on your local machine.
+Currently, this cookbook features [Python Packages](#sec-pythonpkg) for working with data supported by NSIDC DAAC.  It also features some [applications](#sec-applications) that are accessible through a web browser or as stand alone packages that need to be installed on your local machine.
 
 The focus on Python not only reflects the expertise of NSIDC DAAC but also reflects the popularity of Python within the Earth and atmospheric science communities.  However, we recognize that many of our users are more familiar with other programming langauges such as R and Matlab.  We hope that we will be able to include these langauges as the Cookbook develops.
 
@@ -14,53 +14,86 @@ Using web or locally-installed applications is a good way to start to discover a
 There are many Python packages available for working with Earth science data.  The packages we use in this Cookbook are an unashamedly opinionated selection; they are the tools we like to use.  We also think that these tools are the easiest to use for the types of data mananaged by NSIDC DAAC.  Most of the tools have been developed so that researchers do not have to worry about the low-level details of accessing and working with often complicated data used in Earth science.  This reduces the amount of code you have to write and also reduces the number of mistakes you will inevitably make.
 
 #### earthaccess
-[`earthaccess`](https://earthaccess.readthedocs.io/en/latest/) is a package to search for and access NASA Earth science data.
+
+[`earthaccess`](https://earthaccess.readthedocs.io/en/latest/) — search for and access NASA Earth science data.
 
 #### xarray
-[`xarray`](https://xarray.dev/) is a package to work with N-dimensional data (e.g `(time,x,y,z)`).
+
+[`xarray`](https://xarray.dev/) — work with N-dimensional labeled data (e.g. `(time, x, y, z)`).
 
 #### rioxarray
-[`rioxarray`](https://corteva.github.io/rioxarray/stable/index.html) is an extentsion to `xarray` that makes data "geospatially-aware".
+
+[`rioxarray`](https://corteva.github.io/rioxarray/stable/index.html) — add geospatial awareness to `xarray`.
 
 #### rasterio
-[`rasterio`](https://rasterio.readthedocs.io/en/stable/intro.html) is a Python geospatial library for working with raster data.
 
-#### Pandas
-[`pandas`](https://pandas.pydata.org/docs/index.html) is a package to work with tabular data (e.g. the kind of data stored in spreadsheets or databases).
+[`rasterio`](https://rasterio.readthedocs.io/en/stable/intro.html) — read, write, and process raster datasets.
+
+#### pandas
+
+[`pandas`](https://pandas.pydata.org/docs/index.html) — handle tabular data (spreadsheets, databases, CSV).
+
+#### geopandas
 
-#### Geopandas
-[`geopandas`](https://geopandas.org/en/stable/) is an extension to `pandas` to work with geospatial data.
+[`geopandas`](https://geopandas.org/en/stable/) — extend `pandas` to work with geospatial vector data.
 
 #### cartopy
-TBD
+
+[`cartopy`](https://scitools.org.uk/cartopy/docs/latest/) — map projections, transformations, and plotting for geospatial data.
 
 #### SlideRule
-TBD
+
+[`SlideRule`](https://github.com/ICESat2-SlideRule/sliderule-python) — on-demand processing and access to ICESat-2 data.
 
 #### icepyx
-TBD
+
+[`icepyx`](https://icepyx.readthedocs.io/en/latest/) — search, subset, and download ICESat-2 data with built-in metadata handling.
 
 #### satpy
-TBD
+
+[`satpy`](https://satpy.readthedocs.io/en/stable/) — read, composite, and visualize meteorological satellite data.
 
 #### dask
-TBD
 
-## Installing Python Packages
+[`dask`](https://www.dask.org/) — scale computations from laptops to clusters with parallel, out-of-core workflows.
+
+### Installing Python Packages
+
+Most Python packages can be installed from the [Python Package Index (PyPI)](https://pypi.org/) using [pip](https://pip.pypa.io/en/stable/). Some scientific and geospatial packages are easier to install with [conda](https://docs.conda.io/), particularly through the [conda-forge community channel](https://conda-forge.org/).
 
+For more details, see the [official Python documentation on installing packages](https://docs.python.org/3/installing/index.html), the [pip documentation](https://pip.pypa.io/en/stable/getting-started/), and the [conda user guide](https://docs.conda.io/projects/conda/en/latest/user-guide/index.html).
 
 ## Applications {#sec-applications}
 
+Applications provide graphical interfaces for exploring, visualizing, and working with Earth science data. Some run directly in a web browser, while others require installation on your local machine. They are useful for quickly examining datasets, generating plots or maps, and learning about data structures before moving on to scripted workflows.
+
 #### Earthdata Search
 
+[`Earthdata Search`](https://search.earthdata.nasa.gov/) — NASA’s primary web application for searching, filtering, and downloading Earth science data.
+
 #### Open Altimetry
 
+[`Open Altimetry`](https://openaltimetry.org/) — a browser-based tool for visualizing and accessing ICESat and ICESat-2 altimetry data.
+
 #### HDFView
 
+[`HDFView`](https://portal.hdfgroup.org/display/support/HDFView) — a desktop application for viewing and editing HDF4 and HDF5 files.
+
 #### Panoply
 
+[`Panoply`](https://www.giss.nasa.gov/tools/panoply/) — a NASA tool for visualizing netCDF, HDF, and GRIB files with quick map and plot options.
+
 #### NCView
 
+[`NCView`](http://meteora.ucsd.edu/~pierce/ncview_home_page.html) — a lightweight utility for quickly viewing netCDF files, especially time series of 2D variables.
+
 #### QGIS
 
-#### ArcGIS
+[`QGIS`](https://qgis.org/) — an open-source geographic information system for analyzing and visualizing spatial data.
+
+#### ArcGIS
+
+[`ArcGIS`](https://www.esri.com/en-us/arcgis/about-arcgis/overview) — a commercial GIS platform for advanced mapping, spatial analysis, and data management.
+
+
+
diff --git a/tutorials/tutorials_overview.qmd b/tutorials/tutorials_overview.qmd
@@ -1,8 +1,16 @@
 ---
-title: "Tutorials"
+title: "Tutorials / Workshops"
 ---
 
-This section of the cookbook contains longer form tutorials providing examples of data search and access within a workflow.
+This section of the cookbook contains longer form tutorials providing examples of data search and access within a workflow. NSIDC is frequently involved in hackweeks and workshops as well. Content from past events can be found in the [NSIDC Data Tutorials Repository](https://github.com/nsidc/NSIDC-Data-Tutorials).
+
+**Current cookbook tutorials:**
+
+- [Direct cloud access of ICESat-2 (*ATL06 Land Ice Height*) data](/tutorials/ATL06-direct-access.ipynb): a notebook that demonstrates searching for cloud-hosted ICESat-2 data and directly accessing Land Ice Height (ATL06) granules from an Amazon Compute Cloud (EC2) instance using the `earthaccess` package. 
+- [Download of ICESat-2 (*ATL07 Sea Ice*) data](/tutorials/working_with_icesat2_sea_ice_data.ipynb): a notebook that demonstrates searching for and accessing ICESat-2 data using the Python `earthaccess` package, reading and visualizing the data using `xarray` and `pandas`, and that uses `matplotlib` and `cartopy` to produce a map of search results.
+
+**Tutorials coming soon:**
+
+- Ideas?
+- *Let us know what you'd like to see!* 
 
---Direct cloud access of ATL06 Land Ice Height data
---Download of ATL07 Sea Ice Data

-Original file line number
+Diff line change
 *.h5
 *.nc
 _freeze
 +.venv/