Skip to content

Instantly share code, notes, and snippets.

@ajelenak

ajelenak/cloud-access-to-hdf5.ipynb

Last active March 12, 2025 19:11
Show Gist options
  • Save ajelenak/db0d9bf14b7ea4c48acf20249e189c80 to your computer and use it in GitHub Desktop.
Save ajelenak/db0d9bf14b7ea4c48acf20249e189c80 to your computer and use it in GitHub Desktop.
Download ZIP
Access HDF5 Files in S3
Display the source blob
Display the rendered blob
Raw
cloud-access-to-hdf5.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Access HDF5 Files in AWS S3 with h5py\n",
"\n",
"This notebook demonstrates how to read data from an HDF5 file stored as a single object in an AWS S3 bucket with just the standard HDF5 library and the h5py Python package. H5py since version 2.9 can [use](http://docs.h5py.org/en/stable/high/file.html#python-file-like-objects) any [bytes-like](https://docs.python.org/3/glossary.html#term-bytes-like-object) object as an HDF5 file. This can be used to access HDF5 files outside of a typical file system, like in a web object store.\n",
"\n",
"Two examples how to create such byte-like objects are presented here: the [fsspec](https://filesystem-spec.readthedocs.io) package, and a bare-bones Python class that implements the required interface.\n",
"\n",
"As an example HDF5 file we are going to use one from the NASA's Terra Basic Fusion dataset which is available from the AWS Open Data program: https://registry.opendata.aws/terrafusion/."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import io\n",
"from urllib.parse import urlparse\n",
"import boto3\n",
"import h5py\n",
"import fsspec"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Software and Their Versions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Python version:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"3.8.1 (default, Jan 5 2020, 21:31:55) \n",
"[Clang 9.1.0 (clang-902.0.39.2)]\n"
]
}
],
"source": [
"import sys\n",
"print(sys.version)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Package versions:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"boto3 v1.11.4\n",
"h5py v2.10.0\n",
"fsspec v0.6.2\n"
]
}
],
"source": [
"for _ in (boto3, h5py, fsspec):\n",
" print(f'{_.__name__} v{_.__version__}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"HDF5 library version:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'1.10.6'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"h5py.version.hdf5_version"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example HDF5 File in AWS S3"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"s3url = 's3://terrafusiondatasampler/P233/TERRA_BF_L1B_O12236_20020406135439_F000_V001.h5'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Data Access with the fsspec Package\n",
"\n",
"Create a file-system object representing the HDF5 file in S3:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"s3f = fsspec.open(s3url, mode='rb', anon=True, default_fill_cache=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use this object in h5py instead of a file name:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"h5f = h5py.File(s3f.open(), mode='r')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"From now on, use h5py as usual.\n",
"\n",
"Root group attributes:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"InputGranules : b'MOP01-20020406-L1V3.50.0.he5,CER_SSF_Terra-FM1-MODIS_Edition4A_400403.2002040613,CER_SSF_Terra-FM2-MODIS_Edition4A_400403.2002040613,CER_SSF_Terra-FM1-MODIS_Edition4A_400403.2002040614,CER_SSF_Terra-FM2-MODIS_Edition4A_400403.2002040614,CER_SSF_Terra-FM1-MODIS_Edition4A_400403.2002040615,CER_SSF_Terra-FM2-MODIS_Edition4A_400403.2002040615,MOD021KM.A2002096.1355.006.2014230180931.hdf,MOD03.A2002096.1355.006.2012253003116.hdf,MOD021KM.A2002096.1400.006.2014230180941.hdf,MOD03.A2002096.1400.006.2012253003159.hdf,MOD021KM.A2002096.1405.006.2014230180948.hdf,MOD03.A2002096.1405.006.2012253003118.hdf,MOD021KM.A2002096.1410.006.2014230180959.hdf,MOD03.A2002096.1410.006.2012253003337.hdf,MOD021KM.A2002096.1415.006.2014230181010.hdf,MOD02HKM.A2002096.1415.006.2014233052132.hdf,MOD02QKM.A2002096.1415.006.2014234015651.hdf,MOD03.A2002096.1415.006.2012253004649.hdf,MOD021KM.A2002096.1420.006.2014230181030.hdf,MOD02HKM.A2002096.1420.006.2014233052150.hdf,MOD02QKM.A2002096.1420.006.2014234015705.hdf,MOD03.A2002096.1420.006.2012253004223.hdf,MOD021KM.A2002096.1425.006.2014230181050.hdf,MOD02HKM.A2002096.1425.006.2014233052208.hdf,MOD02QKM.A2002096.1425.006.2014234015721.hdf,MOD03.A2002096.1425.006.2012253003743.hdf,MOD021KM.A2002096.1430.006.2014230181106.hdf,MOD02HKM.A2002096.1430.006.2014233052227.hdf,MOD02QKM.A2002096.1430.006.2014234015737.hdf,MOD03.A2002096.1430.006.2012253003821.hdf,MOD021KM.A2002096.1435.006.2014230181127.hdf,MOD02HKM.A2002096.1435.006.2014233052247.hdf,MOD02QKM.A2002096.1435.006.2014234015753.hdf,MOD03.A2002096.1435.006.2012253003715.hdf,MOD021KM.A2002096.1440.006.2014230181144.hdf,MOD02HKM.A2002096.1440.006.2014233052304.hdf,MOD02QKM.A2002096.1440.006.2014234015809.hdf,MOD03.A2002096.1440.006.2012253004157.hdf,MOD021KM.A2002096.1445.006.2014230181203.hdf,MOD02HKM.A2002096.1445.006.2014233052321.hdf,MOD02QKM.A2002096.1445.006.2014234015825.hdf,MOD03.A2002096.1445.006.2012253003532.hdf,MOD021KM.A2002096.1450.006.2014230181221.hdf,MOD02HKM.A2002096.1450.006.2014233052339.hdf,MOD02QKM.A2002096.1450.006.2014234015841.hdf,MOD03.A2002096.1450.006.2012253003515.hdf,MOD021KM.A2002096.1455.006.2014230181242.hdf,MOD02HKM.A2002096.1455.006.2014233052358.hdf,MOD02QKM.A2002096.1455.006.2014234015857.hdf,MOD03.A2002096.1455.006.2012253003613.hdf,MOD021KM.A2002096.1500.006.2014230181303.hdf,MOD02HKM.A2002096.1500.006.2014233052416.hdf,MOD02QKM.A2002096.1500.006.2014234015914.hdf,MOD03.A2002096.1500.006.2012253004353.hdf,MOD021KM.A2002096.1505.006.2014230181319.hdf,MOD02HKM.A2002096.1505.006.2014233052431.hdf,MOD02QKM.A2002096.1505.006.2014234015930.hdf,MOD03.A2002096.1505.006.2012253005511.hdf,MOD021KM.A2002096.1510.006.2014230181331.hdf,MOD03.A2002096.1510.006.2012253003711.hdf,MOD021KM.A2002096.1515.006.2014230181341.hdf,MOD03.A2002096.1515.006.2012253003307.hdf,MOD021KM.A2002096.1520.006.2014230181351.hdf,MOD03.A2002096.1520.006.2012253003123.hdf,MOD021KM.A2002096.1525.006.2014230181358.hdf,MOD03.A2002096.1525.006.2012253003006.hdf,MOD021KM.A2002096.1530.006.2014230181408.hdf,MOD03.A2002096.1530.006.2012253003054.hdf,AST_L1T_00304062002140235_20170807064026_2045.hdf,AST_L1T_00304062002143925_20150422005606_115743.hdf,AST_L1T_00304062002143934_20150422005626_37492.hdf,AST_L1T_00304062002143943_20150422005616_116180.hdf,AST_L1T_00304062002143952_20150422005616_116176.hdf,AST_L1T_00304062002144001_20150422005616_116182.hdf,AST_L1T_00304062002144009_20150422005626_116643.hdf,AST_L1T_00304062002144018_20150422005623_116530.hdf,AST_L1T_00304062002144027_20150422005626_116649.hdf,AST_L1T_00304062002144036_20150422005626_116654.hdf,AST_L1T_00304062002144045_20150422005626_116656.hdf,AST_L1T_00304062002144054_20150422005623_116533.hdf,AST_L1T_00304062002144102_20150422005623_116535.hdf,AST_L1T_00304062002144111_20150424182018_25409.hdf,AST_L1T_00304062002144120_20150422005633_116742.hdf,AST_L1T_00304062002144129_20150424144017_45562.hdf,AST_L1T_00304062002144138_20150424182008_24053.hdf,AST_L1T_00304062002144147_20150422005633_116746.hdf,AST_L1T_00304062002144156_20150424144019_45631.hdf,AST_L1T_00304062002144204_20150424144019_45636.hdf,AST_L1T_00304062002144213_20150424144019_45638.hdf,AST_L1T_00304062002144222_20150424144016_79056.hdf,AST_L1T_00304062002144231_20150424144023_38934.hdf,AST_L1T_00304062002144240_20150422005633_116747.hdf,AST_L1T_00304062002144249_20150424144037_46143.hdf,AST_L1T_00304062002144258_20150422005704_56770.hdf,AST_L1T_00304062002144306_20150424144037_46145.hdf,AST_L1T_00304062002144315_20150422005641_121992.hdf,AST_L1T_00304062002144324_20150424144039_44739.hdf,AST_L1T_00304062002144333_20150422005634_37673.hdf,AST_L1T_00304062002144342_20150424144039_44766.hdf,AST_L1T_00304062002144351_20150424144033_39207.hdf,AST_L1T_00304062002144359_20150424144033_39202.hdf,AST_L1T_00304062002145038_20150422005646_38311.hdf,AST_L1T_00304062002145047_20150422005646_117999.hdf,AST_L1T_00304062002145056_20150422005714_56935.hdf,AST_L1T_00304062002145105_20150422005714_56938.hdf,AST_L1T_00304062002145114_20150422005654_38591.hdf,AST_L1T_00304062002145122_20150422005643_117619.hdf,AST_L1T_00304062002145131_20150422005653_118723.hdf,AST_L1T_00304062002145140_20150422005711_559.hdf,AST_L1T_00304062002145149_20150422005706_119709.hdf,AST_L1T_00304062002145158_20150422005653_118724.hdf,AST_L1T_00304062002145207_20150422005706_39171.hdf,AST_L1T_00304062002145216_20150422005703_119401.hdf,AST_L1T_00304062002145224_20150422005714_56940.hdf,AST_L1T_00304062002145233_20150422005716_40130.hdf,AST_L1T_00304062002145242_20150422005716_40132.hdf,AST_L1T_00304062002145251_20150424144039_46270.hdf,AST_L1T_00304062002145300_20150422005724_40466.hdf,AST_L1T_00304062002145309_20150424144036_79756.hdf,AST_L1T_00304062002145318_20150422005724_40471.hdf,AST_L1T_00304062002145327_20150422005724_40478.hdf,AST_L1T_00304062002145335_20150422005724_40482.hdf,AST_L1T_00304062002145344_20150422005714_39803.hdf,AST_L1T_00304062002145353_20150422005714_39805.hdf,AST_L1T_00304062002145402_20150422005724_40484.hdf,AST_L1T_00304062002145411_20150422005726_40633.hdf,AST_L1T_00304062002145420_20150422005726_40639.hdf,AST_L1T_00304062002145429_20150422005726_40641.hdf,AST_L1T_00304062002145437_20150422005733_1767.hdf,AST_L1T_00304062002145446_20150422005733_1770.hdf,AST_L1T_00304062002145455_20150422005723_901.hdf,AST_L1T_00304062002145504_20150422005736_41422.hdf,AST_L1T_00304062002145513_20150422005746_42745.hdf,AST_L1T_00304062002145522_20150422005736_58385.hdf,AST_L1T_00304062002145531_20150422005751_2995.hdf,AST_L1T_00304062002145540_20150422005801_4120.hdf,AST_L1T_00304062002145548_20150422005801_4122.hdf,AST_L1T_00304062002145557_20150422005743_2330.hdf,AST_L1T_00304062002145606_20150422005751_3000.hdf,AST_L1T_00304062002145615_20150422005743_2333.hdf,AST_L1T_00304062002145624_20150422005744_42519.hdf,AST_L1T_00304062002145633_20150422005754_59024.hdf,AST_L1T_00304062002145642_20150422005754_59029.hdf,AST_L1T_00304062002145650_20150422005754_59034.hdf,AST_L1T_00304062002145659_20150422005744_58702.hdf,AST_L1T_00304062002145708_20150422005744_58707.hdf,AST_L1T_00304062002145717_20150422005744_58711.hdf,AST_L1T_00304062002145726_20150422005746_58825.hdf,AST_L1T_00304062002145735_20150422005746_58828.hdf,AST_L1T_00304062002145744_20150422005806_60537.hdf,AST_L1T_00304062002145753_20150422005806_60543.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_AA_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_AF_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_AN_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_BA_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_BF_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_CA_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_CF_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_DA_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_DF_F03_0024.hdf,MISR_AM1_AGP_P233_F01_24.hdf,MISR_AM1_GP_GMP_P233_O012236_F03_0013.hdf,MISR_HRLL_P233.hdf,'\n",
"acknowledgement : b'This data set was created with funding from NASA ACCESS Grant #NNX16AM07A.'\n",
"contributor_email : b'[email protected], [email protected]'\n",
"contributor_name : b'Muqun Yang, Landon Clipp, Yizhao Gao, Guangyu Zhao, Larry Di Girolamo'\n",
"institution : b'The HDF Group, University of Illinois'\n",
"keywords : b'Earth Science, TERRA, MISR, MODIS, ASTER,CERES,MOPITT, Radiance'\n",
"license : b'No constraints on data access or use'\n",
"platform : b'TERRA'\n",
"processing_level : b'Level 1'\n",
"product_version : b'V001'\n",
"project : b'NASA ACCESS to Terra Fusion Product'\n",
"sensors : b'MODIS,MISR,ASTER,CERES,MOPITT'\n",
"summary : b'A Terra level-1 radiance basic fusion product that combines calibrated radiance measurements from MISR, MODIS, ASTER, CERES and MOPITT'\n",
"title : b'TERRA L1B Radiance Basic Fusion Product'\n",
"version : b'First version of this product'\n"
]
}
],
"source": [
"for n, v in h5f.attrs.items():\n",
" print(f'{n} : {v}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"List ten HDF5 groups or datasets in the root group:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/AN_XDim_BlueBand is <class 'h5py._hl.dataset.Dataset'>\n",
"/AN_XDim_GreenBand is <class 'h5py._hl.dataset.Dataset'>\n",
"/AN_XDim_NIRBand is <class 'h5py._hl.dataset.Dataset'>\n",
"/AN_YDim_BlueBand is <class 'h5py._hl.dataset.Dataset'>\n",
"/AN_YDim_GreenBand is <class 'h5py._hl.dataset.Dataset'>\n",
"/AN_YDim_NIRBand is <class 'h5py._hl.dataset.Dataset'>\n",
"/ASTER is <class 'h5py._hl.group.Group'>\n",
"/ASTER_PointingAngleDim is <class 'h5py._hl.dataset.Dataset'>\n",
"/ASTER_Solar_GeometryDim is <class 'h5py._hl.dataset.Dataset'>\n",
"/Band_1KM_Emissive_MODIS_SWATH_Type_L1B is <class 'h5py._hl.dataset.Dataset'>\n",
"/Band_1KM_RefSB_MODIS_SWATH_Type_L1B is <class 'h5py._hl.dataset.Dataset'>\n",
"-=- end -=-\n"
]
}
],
"source": [
"for count, h5o in enumerate(h5f.values()):\n",
" print(f'{h5o.name} is {type(h5o)}')\n",
" if count > 9:\n",
" print('-=- end -=-')\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's select one of the HDF5 datasets in the file:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<HDF5 dataset \"Band_1KM_Emissive_MODIS_SWATH_Type_L1B\": shape (16,), type \"<f4\">"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dset = h5f['Band_1KM_Emissive_MODIS_SWATH_Type_L1B']\n",
"dset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The dataset's values are:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([20., 21., 22., 23., 24., 25., 27., 28., 29., 30., 31., 32., 33.,\n",
" 34., 35., 36.], dtype=float32)"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dset[...]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When done:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"h5f.close()\n",
"s3f.close()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Data Access with a Custom Python Class\n",
"\n",
"The class `S3File` below implements a minimal interface required by h5py to access HDF5 files in S3. It is included here as an example of the bytes-like API. Logging HDF5 library's data reads (as [HTTP range GET requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests)) is just an added bonus that can be very useful for benchmarking.\n",
"\n",
"When logging is enabled with `S3File(..., log=True)`, a message of the form `GET bytes={start byte}-{end byte} SIZE {size}` will be printed every time HDF5 library makes a data read operation. The `start byte` and `end byte` indicate the start and end of a block of bytes in the HDF5 file and `size` is the number of read bytes (`{end byte}` - `{start byte}` + 1)."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"class S3File(io.RawIOBase):\n",
"\n",
" def __init__(self, s3url, log=False):\n",
" self._cursor = 0\n",
" purl = urlparse(s3url)\n",
" self._s3obj = boto3.resource(purl.scheme).Object(purl.netloc, \n",
" purl.path[1:])\n",
" self._log = log\n",
"\n",
" def __repr__(self):\n",
" bucket = self._s3obj.bucket_name\n",
" key = self._s3obj.key\n",
" return f'<{type(self).__name__} s3://{bucket}/{key} at 0x{id(self):x}>'\n",
"\n",
" def readable(self):\n",
" return True\n",
"\n",
" def seekable(self):\n",
" return True\n",
"\n",
" @property\n",
" def size(self):\n",
" return self._s3obj.content_length\n",
"\n",
" def tell(self):\n",
" return self._cursor\n",
"\n",
" def seek(self, offset, whence=io.SEEK_SET):\n",
" if whence == io.SEEK_SET:\n",
" self._cursor = offset\n",
" elif whence == io.SEEK_CUR:\n",
" self._cursor += offset\n",
" elif whence == io.SEEK_END:\n",
" self._cursor = self.size + offset\n",
" else:\n",
" raise ValueError(f'{whence}: Unknown whence value')\n",
" return self._cursor\n",
"\n",
" def read(self, size=-1):\n",
" if size == -1:\n",
" range_header = f'bytes={self._cursor}-'\n",
" self.seek(offset=0, whence=io.SEEK_END)\n",
" else:\n",
" new_pos = self._cursor + size\n",
" if new_pos >= self.size:\n",
" return self.read()\n",
"\n",
" range_header = f'bytes={self._cursor}-{new_pos - 1}'\n",
" self.seek(offset=size, whence=io.SEEK_CUR)\n",
" if self._log:\n",
" print(f'GET {range_header} SIZE {size}')\n",
" return self._s3obj.get(Range=range_header)[\"Body\"].read()\n",
"\n",
" def readinto(self, buff):\n",
" data = self.read(len(buff))\n",
" buff[:len(data)] = data\n",
" return len(data)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We are going to repeat the data access examples from the above but, first, create an object that represents the HDF5 file in S3 and enable logging of HDF5 library data reads:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<S3File s3://terrafusiondatasampler/P233/TERRA_BF_L1B_O12236_20020406135439_F000_V001.h5 at 0x10bb8a6d0>"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"s3f = S3File(s3url, log=True)\n",
"s3f"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Supply this object to h5py, and proceed as usual:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"GET bytes=0-7 SIZE 8\n",
"GET bytes=0-15 SIZE 16\n",
"GET bytes=16-95 SIZE 80\n",
"GET bytes=96-607 SIZE 512\n",
"GET bytes=54307049708-54307049931 SIZE 224\n",
"GET bytes=54307974340-54307974651 SIZE 312\n",
"GET bytes=54612761529-54612761640 SIZE 112\n",
"GET bytes=54308713668-54308714331 SIZE 664\n",
"GET bytes=54307975036-54307975139 SIZE 104\n",
"GET bytes=54614245337- SIZE -1\n"
]
}
],
"source": [
"h5f = h5py.File(s3f, mode='r')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Each _GET_ line above represents a single HDF5 library data read from a \"file\" which the Python class executed as one HTTP range GET request to the S3 object.\n",
"\n",
"Root group attributes:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"InputGranules : b'MOP01-20020406-L1V3.50.0.he5,CER_SSF_Terra-FM1-MODIS_Edition4A_400403.2002040613,CER_SSF_Terra-FM2-MODIS_Edition4A_400403.2002040613,CER_SSF_Terra-FM1-MODIS_Edition4A_400403.2002040614,CER_SSF_Terra-FM2-MODIS_Edition4A_400403.2002040614,CER_SSF_Terra-FM1-MODIS_Edition4A_400403.2002040615,CER_SSF_Terra-FM2-MODIS_Edition4A_400403.2002040615,MOD021KM.A2002096.1355.006.2014230180931.hdf,MOD03.A2002096.1355.006.2012253003116.hdf,MOD021KM.A2002096.1400.006.2014230180941.hdf,MOD03.A2002096.1400.006.2012253003159.hdf,MOD021KM.A2002096.1405.006.2014230180948.hdf,MOD03.A2002096.1405.006.2012253003118.hdf,MOD021KM.A2002096.1410.006.2014230180959.hdf,MOD03.A2002096.1410.006.2012253003337.hdf,MOD021KM.A2002096.1415.006.2014230181010.hdf,MOD02HKM.A2002096.1415.006.2014233052132.hdf,MOD02QKM.A2002096.1415.006.2014234015651.hdf,MOD03.A2002096.1415.006.2012253004649.hdf,MOD021KM.A2002096.1420.006.2014230181030.hdf,MOD02HKM.A2002096.1420.006.2014233052150.hdf,MOD02QKM.A2002096.1420.006.2014234015705.hdf,MOD03.A2002096.1420.006.2012253004223.hdf,MOD021KM.A2002096.1425.006.2014230181050.hdf,MOD02HKM.A2002096.1425.006.2014233052208.hdf,MOD02QKM.A2002096.1425.006.2014234015721.hdf,MOD03.A2002096.1425.006.2012253003743.hdf,MOD021KM.A2002096.1430.006.2014230181106.hdf,MOD02HKM.A2002096.1430.006.2014233052227.hdf,MOD02QKM.A2002096.1430.006.2014234015737.hdf,MOD03.A2002096.1430.006.2012253003821.hdf,MOD021KM.A2002096.1435.006.2014230181127.hdf,MOD02HKM.A2002096.1435.006.2014233052247.hdf,MOD02QKM.A2002096.1435.006.2014234015753.hdf,MOD03.A2002096.1435.006.2012253003715.hdf,MOD021KM.A2002096.1440.006.2014230181144.hdf,MOD02HKM.A2002096.1440.006.2014233052304.hdf,MOD02QKM.A2002096.1440.006.2014234015809.hdf,MOD03.A2002096.1440.006.2012253004157.hdf,MOD021KM.A2002096.1445.006.2014230181203.hdf,MOD02HKM.A2002096.1445.006.2014233052321.hdf,MOD02QKM.A2002096.1445.006.2014234015825.hdf,MOD03.A2002096.1445.006.2012253003532.hdf,MOD021KM.A2002096.1450.006.2014230181221.hdf,MOD02HKM.A2002096.1450.006.2014233052339.hdf,MOD02QKM.A2002096.1450.006.2014234015841.hdf,MOD03.A2002096.1450.006.2012253003515.hdf,MOD021KM.A2002096.1455.006.2014230181242.hdf,MOD02HKM.A2002096.1455.006.2014233052358.hdf,MOD02QKM.A2002096.1455.006.2014234015857.hdf,MOD03.A2002096.1455.006.2012253003613.hdf,MOD021KM.A2002096.1500.006.2014230181303.hdf,MOD02HKM.A2002096.1500.006.2014233052416.hdf,MOD02QKM.A2002096.1500.006.2014234015914.hdf,MOD03.A2002096.1500.006.2012253004353.hdf,MOD021KM.A2002096.1505.006.2014230181319.hdf,MOD02HKM.A2002096.1505.006.2014233052431.hdf,MOD02QKM.A2002096.1505.006.2014234015930.hdf,MOD03.A2002096.1505.006.2012253005511.hdf,MOD021KM.A2002096.1510.006.2014230181331.hdf,MOD03.A2002096.1510.006.2012253003711.hdf,MOD021KM.A2002096.1515.006.2014230181341.hdf,MOD03.A2002096.1515.006.2012253003307.hdf,MOD021KM.A2002096.1520.006.2014230181351.hdf,MOD03.A2002096.1520.006.2012253003123.hdf,MOD021KM.A2002096.1525.006.2014230181358.hdf,MOD03.A2002096.1525.006.2012253003006.hdf,MOD021KM.A2002096.1530.006.2014230181408.hdf,MOD03.A2002096.1530.006.2012253003054.hdf,AST_L1T_00304062002140235_20170807064026_2045.hdf,AST_L1T_00304062002143925_20150422005606_115743.hdf,AST_L1T_00304062002143934_20150422005626_37492.hdf,AST_L1T_00304062002143943_20150422005616_116180.hdf,AST_L1T_00304062002143952_20150422005616_116176.hdf,AST_L1T_00304062002144001_20150422005616_116182.hdf,AST_L1T_00304062002144009_20150422005626_116643.hdf,AST_L1T_00304062002144018_20150422005623_116530.hdf,AST_L1T_00304062002144027_20150422005626_116649.hdf,AST_L1T_00304062002144036_20150422005626_116654.hdf,AST_L1T_00304062002144045_20150422005626_116656.hdf,AST_L1T_00304062002144054_20150422005623_116533.hdf,AST_L1T_00304062002144102_20150422005623_116535.hdf,AST_L1T_00304062002144111_20150424182018_25409.hdf,AST_L1T_00304062002144120_20150422005633_116742.hdf,AST_L1T_00304062002144129_20150424144017_45562.hdf,AST_L1T_00304062002144138_20150424182008_24053.hdf,AST_L1T_00304062002144147_20150422005633_116746.hdf,AST_L1T_00304062002144156_20150424144019_45631.hdf,AST_L1T_00304062002144204_20150424144019_45636.hdf,AST_L1T_00304062002144213_20150424144019_45638.hdf,AST_L1T_00304062002144222_20150424144016_79056.hdf,AST_L1T_00304062002144231_20150424144023_38934.hdf,AST_L1T_00304062002144240_20150422005633_116747.hdf,AST_L1T_00304062002144249_20150424144037_46143.hdf,AST_L1T_00304062002144258_20150422005704_56770.hdf,AST_L1T_00304062002144306_20150424144037_46145.hdf,AST_L1T_00304062002144315_20150422005641_121992.hdf,AST_L1T_00304062002144324_20150424144039_44739.hdf,AST_L1T_00304062002144333_20150422005634_37673.hdf,AST_L1T_00304062002144342_20150424144039_44766.hdf,AST_L1T_00304062002144351_20150424144033_39207.hdf,AST_L1T_00304062002144359_20150424144033_39202.hdf,AST_L1T_00304062002145038_20150422005646_38311.hdf,AST_L1T_00304062002145047_20150422005646_117999.hdf,AST_L1T_00304062002145056_20150422005714_56935.hdf,AST_L1T_00304062002145105_20150422005714_56938.hdf,AST_L1T_00304062002145114_20150422005654_38591.hdf,AST_L1T_00304062002145122_20150422005643_117619.hdf,AST_L1T_00304062002145131_20150422005653_118723.hdf,AST_L1T_00304062002145140_20150422005711_559.hdf,AST_L1T_00304062002145149_20150422005706_119709.hdf,AST_L1T_00304062002145158_20150422005653_118724.hdf,AST_L1T_00304062002145207_20150422005706_39171.hdf,AST_L1T_00304062002145216_20150422005703_119401.hdf,AST_L1T_00304062002145224_20150422005714_56940.hdf,AST_L1T_00304062002145233_20150422005716_40130.hdf,AST_L1T_00304062002145242_20150422005716_40132.hdf,AST_L1T_00304062002145251_20150424144039_46270.hdf,AST_L1T_00304062002145300_20150422005724_40466.hdf,AST_L1T_00304062002145309_20150424144036_79756.hdf,AST_L1T_00304062002145318_20150422005724_40471.hdf,AST_L1T_00304062002145327_20150422005724_40478.hdf,AST_L1T_00304062002145335_20150422005724_40482.hdf,AST_L1T_00304062002145344_20150422005714_39803.hdf,AST_L1T_00304062002145353_20150422005714_39805.hdf,AST_L1T_00304062002145402_20150422005724_40484.hdf,AST_L1T_00304062002145411_20150422005726_40633.hdf,AST_L1T_00304062002145420_20150422005726_40639.hdf,AST_L1T_00304062002145429_20150422005726_40641.hdf,AST_L1T_00304062002145437_20150422005733_1767.hdf,AST_L1T_00304062002145446_20150422005733_1770.hdf,AST_L1T_00304062002145455_20150422005723_901.hdf,AST_L1T_00304062002145504_20150422005736_41422.hdf,AST_L1T_00304062002145513_20150422005746_42745.hdf,AST_L1T_00304062002145522_20150422005736_58385.hdf,AST_L1T_00304062002145531_20150422005751_2995.hdf,AST_L1T_00304062002145540_20150422005801_4120.hdf,AST_L1T_00304062002145548_20150422005801_4122.hdf,AST_L1T_00304062002145557_20150422005743_2330.hdf,AST_L1T_00304062002145606_20150422005751_3000.hdf,AST_L1T_00304062002145615_20150422005743_2333.hdf,AST_L1T_00304062002145624_20150422005744_42519.hdf,AST_L1T_00304062002145633_20150422005754_59024.hdf,AST_L1T_00304062002145642_20150422005754_59029.hdf,AST_L1T_00304062002145650_20150422005754_59034.hdf,AST_L1T_00304062002145659_20150422005744_58702.hdf,AST_L1T_00304062002145708_20150422005744_58707.hdf,AST_L1T_00304062002145717_20150422005744_58711.hdf,AST_L1T_00304062002145726_20150422005746_58825.hdf,AST_L1T_00304062002145735_20150422005746_58828.hdf,AST_L1T_00304062002145744_20150422005806_60537.hdf,AST_L1T_00304062002145753_20150422005806_60543.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_AA_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_AF_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_AN_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_BA_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_BF_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_CA_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_CF_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_DA_F03_0024.hdf,MISR_AM1_GRP_ELLIPSOID_GM_P233_O012236_DF_F03_0024.hdf,MISR_AM1_AGP_P233_F01_24.hdf,MISR_AM1_GP_GMP_P233_O012236_F03_0013.hdf,MISR_HRLL_P233.hdf,'\n",
"acknowledgement : b'This data set was created with funding from NASA ACCESS Grant #NNX16AM07A.'\n",
"contributor_email : b'[email protected], [email protected]'\n",
"contributor_name : b'Muqun Yang, Landon Clipp, Yizhao Gao, Guangyu Zhao, Larry Di Girolamo'\n",
"institution : b'The HDF Group, University of Illinois'\n",
"keywords : b'Earth Science, TERRA, MISR, MODIS, ASTER,CERES,MOPITT, Radiance'\n",
"license : b'No constraints on data access or use'\n",
"platform : b'TERRA'\n",
"processing_level : b'Level 1'\n",
"product_version : b'V001'\n",
"project : b'NASA ACCESS to Terra Fusion Product'\n",
"sensors : b'MODIS,MISR,ASTER,CERES,MOPITT'\n",
"summary : b'A Terra level-1 radiance basic fusion product that combines calibrated radiance measurements from MISR, MODIS, ASTER, CERES and MOPITT'\n",
"title : b'TERRA L1B Radiance Basic Fusion Product'\n",
"version : b'First version of this product'\n"
]
}
],
"source": [
"for n, v in h5f.attrs.items():\n",
" print(f'{n} : {v}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"List ten HDF5 groups or datasets in the root group:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"GET bytes=136-679 SIZE 544\n",
"GET bytes=21031789191-21031789734 SIZE 544\n",
"GET bytes=1504-1831 SIZE 328\n",
"GET bytes=52502566307-52502566634 SIZE 328\n",
"GET bytes=52270697465-52270697792 SIZE 328\n",
"GET bytes=717405856-717406183 SIZE 328\n",
"GET bytes=24893600-24893927 SIZE 328\n",
"GET bytes=22738861009-22738861552 SIZE 544\n",
"GET bytes=13611877761-13611878088 SIZE 328\n",
"GET bytes=15512458108-15512458435 SIZE 328\n",
"GET bytes=17326271654-17326271981 SIZE 328\n",
"GET bytes=19158465702-19158466029 SIZE 328\n",
"GET bytes=21031785615-21031785942 SIZE 328\n",
"GET bytes=22738863409-22738863736 SIZE 328\n",
"GET bytes=24468475525-24468475852 SIZE 328\n",
"GET bytes=26205428488-26205428815 SIZE 328\n",
"GET bytes=27976565801-27976566128 SIZE 328\n",
"GET bytes=29852218777-29852219104 SIZE 328\n",
"GET bytes=31712232321-31712232648 SIZE 328\n",
"GET bytes=33538428326-33538428653 SIZE 328\n",
"GET bytes=35249605148-35249605475 SIZE 328\n",
"GET bytes=37008974014-37008974341 SIZE 328\n",
"GET bytes=38709752416-38709752743 SIZE 328\n",
"GET bytes=40474759223-40474759550 SIZE 328\n",
"GET bytes=42262139839-42262140166 SIZE 328\n",
"GET bytes=44050568312-44050568639 SIZE 328\n",
"GET bytes=45848175822-45848176149 SIZE 328\n",
"GET bytes=47744744339-47744744666 SIZE 328\n",
"GET bytes=49668368559-49668368886 SIZE 328\n",
"GET bytes=13036753742-13036754069 SIZE 328\n",
"GET bytes=15512460812-15512461139 SIZE 328\n",
"GET bytes=38709752744-38709753287 SIZE 544\n",
"GET bytes=17326274494-17326274821 SIZE 328\n",
"GET bytes=19158472710-19158473037 SIZE 328\n",
"GET bytes=21031788319-21031788646 SIZE 328\n",
"GET bytes=22738866249-22738866576 SIZE 328\n",
"GET bytes=24468478365-24468478692 SIZE 328\n",
"GET bytes=26205431328-26205431655 SIZE 328\n",
"GET bytes=27976568505-27976568832 SIZE 328\n",
"GET bytes=29852225721-29852226048 SIZE 328\n",
"GET bytes=31712235161-31712235488 SIZE 328\n",
"GET bytes=33538431166-33538431493 SIZE 328\n",
"GET bytes=35249612092-35249612419 SIZE 328\n",
"GET bytes=37008976854-37008977181 SIZE 328\n",
"GET bytes=38709755800-38709756127 SIZE 328\n",
"GET bytes=40474766231-40474766558 SIZE 328\n",
"GET bytes=42262142543-42262142870 SIZE 328\n",
"GET bytes=44050571152-44050571479 SIZE 328\n",
"GET bytes=45848182622-45848182949 SIZE 328\n",
"GET bytes=47744747179-47744747506 SIZE 328\n",
"GET bytes=49668371399-49668371726 SIZE 328\n",
"GET bytes=14086521548-14086521875 SIZE 328\n",
"GET bytes=17326269470-17326269797 SIZE 328\n",
"GET bytes=19158463518-19158463845 SIZE 328\n",
"GET bytes=21031783431-21031783758 SIZE 328\n",
"GET bytes=23613132789-23613133332 SIZE 544\n",
"GET bytes=22738860681-22738861008 SIZE 328\n",
"GET bytes=24468473341-24468473668 SIZE 328\n",
"GET bytes=26205426304-26205426631 SIZE 328\n",
"GET bytes=27976563617-27976563944 SIZE 328\n",
"GET bytes=29852216497-29852216824 SIZE 328\n",
"GET bytes=31712230137-31712230464 SIZE 328\n",
"GET bytes=33538426046-33538426373 SIZE 328\n",
"GET bytes=35249602964-35249603291 SIZE 328\n",
"GET bytes=37008971734-37008972061 SIZE 328\n",
"GET bytes=38709750136-38709750463 SIZE 328\n",
"GET bytes=40474756943-40474757270 SIZE 328\n",
"GET bytes=42262137655-42262137982 SIZE 328\n",
"GET bytes=44050566128-44050566455 SIZE 328\n",
"GET bytes=45848173638-45848173965 SIZE 328\n",
"GET bytes=47744742059-47744742386 SIZE 328\n",
"GET bytes=49668366279-49668366606 SIZE 328\n",
"GET bytes=15040283196-15040283523 SIZE 328\n",
"GET bytes=16882197749-16882198076 SIZE 328\n",
"GET bytes=18690357984-18690358311 SIZE 328\n",
"GET bytes=45394230369-45394230912 SIZE 544\n",
"GET bytes=20576256421-20576256748 SIZE 328\n",
"GET bytes=22310318245-22310318572 SIZE 328\n",
"GET bytes=24041807599-24041807926 SIZE 328\n",
"GET bytes=25755895169-25755895496 SIZE 328\n",
"GET bytes=27507842135-27507842462 SIZE 328\n",
"GET bytes=29381326939-29381327266 SIZE 328\n",
"GET bytes=31249439121-31249439448 SIZE 328\n",
"GET bytes=33107249608-33107249935 SIZE 328\n",
"GET bytes=34822156332-34822156659 SIZE 328\n",
"GET bytes=36569233233-36569233560 SIZE 328\n",
"GET bytes=38293919500-38293919827 SIZE 328\n",
"GET bytes=40025175908-40025176235 SIZE 328\n",
"GET bytes=41816952302-41816952629 SIZE 328\n",
"GET bytes=43602258569-43602258896 SIZE 328\n",
"GET bytes=45394230041-45394230368 SIZE 328\n",
"GET bytes=47266564610-47266564937 SIZE 328\n",
"GET bytes=49187981825-49187982152 SIZE 328\n",
"GET bytes=13137971605-13137971932 SIZE 328\n",
"GET bytes=16436120476-16436120803 SIZE 328\n",
"GET bytes=30783998081-30783998624 SIZE 544\n",
"GET bytes=18222490060-18222490387 SIZE 328\n",
"GET bytes=20102614157-20102614484 SIZE 328\n",
"GET bytes=21884012585-21884012912 SIZE 328\n",
"GET bytes=23613137469-23613137796 SIZE 328\n",
"GET bytes=25313180797-25313181124 SIZE 328\n",
"GET bytes=27056504922-27056505249 SIZE 328\n",
"GET bytes=28912562065-28912562392 SIZE 328\n",
"GET bytes=30784002761-30784003088 SIZE 328\n",
"GET bytes=32653280725-32653281052 SIZE 328\n",
"GET bytes=34394019875-34394020202 SIZE 328\n",
"GET bytes=36121976181-36121976508 SIZE 328\n",
"GET bytes=37876154212-37876154539 SIZE 328\n",
"GET bytes=39576274957-39576275284 SIZE 328\n",
"GET bytes=41368035049-41368035376 SIZE 328\n",
"GET bytes=43157906762-43157907089 SIZE 328\n",
"GET bytes=44947464383-44947464710 SIZE 328\n",
"GET bytes=46791511651-46791511978 SIZE 328\n",
"GET bytes=48704798350-48704798677 SIZE 328\n",
"GET bytes=14563520409-14563520736 SIZE 328\n",
"GET bytes=44947459463-44947460006 SIZE 544\n",
"GET bytes=16436116012-16436116339 SIZE 328\n",
"GET bytes=18222485596-18222485923 SIZE 328\n",
"GET bytes=20102609693-20102610020 SIZE 328\n",
"GET bytes=21884008121-21884008448 SIZE 328\n",
"GET bytes=23613132461-23613132788 SIZE 328\n",
"GET bytes=25313176333-25313176660 SIZE 328\n",
"GET bytes=27056500458-27056500785 SIZE 328\n",
"GET bytes=28912557601-28912557928 SIZE 328\n",
"GET bytes=30783997753-30783998080 SIZE 328\n",
"GET bytes=32653276021-32653276348 SIZE 328\n",
"GET bytes=34394015411-34394015738 SIZE 328\n",
"GET bytes=36121971717-36121972044 SIZE 328\n",
"GET bytes=37876149748-37876150075 SIZE 328\n",
"GET bytes=39576270253-39576270580 SIZE 328\n",
"GET bytes=41368030345-41368030672 SIZE 328\n",
"GET bytes=43157902058-43157902385 SIZE 328\n",
"GET bytes=44947459135-44947459462 SIZE 328\n",
"GET bytes=46791506947-46791507274 SIZE 328\n",
"GET bytes=48704793646-48704793973 SIZE 328\n",
"GET bytes=51184730894-51184731221 SIZE 328\n",
"GET bytes=51474520404-51474520731 SIZE 328\n",
"GET bytes=51803412645-51803412972 SIZE 328\n",
"GET bytes=53058309896-53058310223 SIZE 328\n",
"GET bytes=53684371416-53684371743 SIZE 328\n",
"GET bytes=54289855269-54289855596 SIZE 328\n",
"GET bytes=21031788647-21031789190 SIZE 544\n",
"GET bytes=137414188-137414515 SIZE 328\n",
"GET bytes=51458613033-51458613360 SIZE 328\n",
"GET bytes=51493215966-51493216293 SIZE 328\n",
"GET bytes=51184354614-51184354941 SIZE 328\n",
"GET bytes=51490989110-51490989437 SIZE 328\n",
"GET bytes=50108651583-50108651910 SIZE 328\n",
"GET bytes=1524937582-1524937909 SIZE 328\n",
"GET bytes=10648876-10649203 SIZE 328\n",
"GET bytes=10447972-10448299 SIZE 328\n",
"GET bytes=680-1191 SIZE 512\n",
"GET bytes=32180655678-32180678205 SIZE 22528\n",
"GET bytes=52270697193-52270697704 SIZE 512\n",
"GET bytes=51768642458-51768642553 SIZE 96\n",
"GET bytes=52270697793-52270697984 SIZE 192\n",
"/AN_XDim_BlueBand is <class 'h5py._hl.dataset.Dataset'>\n",
"GET bytes=52502556443-52502556954 SIZE 512\n",
"/AN_XDim_GreenBand is <class 'h5py._hl.dataset.Dataset'>\n",
"GET bytes=52502566035-52502566546 SIZE 512\n",
"GET bytes=51784655798-51784655893 SIZE 96\n",
"GET bytes=51801183213-51801183404 SIZE 192\n",
"/AN_XDim_NIRBand is <class 'h5py._hl.dataset.Dataset'>\n",
"GET bytes=52270697985-52270698496 SIZE 512\n",
"/AN_YDim_BlueBand is <class 'h5py._hl.dataset.Dataset'>\n",
"GET bytes=52502556923-52502557434 SIZE 512\n",
"/AN_YDim_GreenBand is <class 'h5py._hl.dataset.Dataset'>\n",
"GET bytes=52743045994-52743046505 SIZE 512\n",
"/AN_YDim_NIRBand is <class 'h5py._hl.dataset.Dataset'>\n",
"GET bytes=12884543500-12884544011 SIZE 512\n",
"GET bytes=12759000353-12759000680 SIZE 328\n",
"GET bytes=12887624136-12887624527 SIZE 392\n",
"/ASTER is <class 'h5py._hl.group.Group'>\n",
"GET bytes=12892121070-12892121581 SIZE 512\n",
"GET bytes=12901557311-12901557534 SIZE 224\n",
"GET bytes=49593690272-49593694399 SIZE 4128\n",
"/ASTER_PointingAngleDim is <class 'h5py._hl.dataset.Dataset'>\n",
"GET bytes=12901557535-12901558046 SIZE 512\n",
"GET bytes=12901558431-12901558654 SIZE 224\n",
"GET bytes=49109938172-49109941003 SIZE 2832\n",
"/ASTER_Solar_GeometryDim is <class 'h5py._hl.dataset.Dataset'>\n",
"GET bytes=137412780-137413291 SIZE 512\n",
"GET bytes=137413212-137413435 SIZE 224\n",
"GET bytes=12758999185-12759000000 SIZE 816\n",
"/Band_1KM_Emissive_MODIS_SWATH_Type_L1B is <class 'h5py._hl.dataset.Dataset'>\n",
"GET bytes=595302314-595302825 SIZE 512\n",
"GET bytes=595302746-595302969 SIZE 224\n",
"GET bytes=10960025998-10960026525 SIZE 528\n",
"/Band_1KM_RefSB_MODIS_SWATH_Type_L1B is <class 'h5py._hl.dataset.Dataset'>\n",
"-=- end -=-\n"
]
}
],
"source": [
"for count, h5o in enumerate(h5f.values()):\n",
" print(f'{h5o.name} is {type(h5o)}')\n",
" if count > 9:\n",
" print('-=- end -=-')\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's select one of the HDF5 datasets in the file:"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<HDF5 dataset \"Band_1KM_Emissive_MODIS_SWATH_Type_L1B\": shape (16,), type \"<f4\">"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dset = h5f['Band_1KM_Emissive_MODIS_SWATH_Type_L1B']\n",
"dset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The dataset's values are:"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"GET bytes=10569992-10570055 SIZE 64\n"
]
},
{
"data": {
"text/plain": [
"array([20., 21., 22., 23., 24., 25., 27., 28., 29., 30., 31., 32., 33.,\n",
" 34., 35., 36.], dtype=float32)"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dset[...]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When done:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"h5f.close()\n",
"s3f.close()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.1"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment