This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Below you find a pydantic model that closely represents Googles product data model for their Vertex AI Search for Retail. | |
Their model is closely described | |
- in their restapi overview: https://cloud.google.com/retail/docs/reference/rest/v2/projects.locations.catalogs.branches.products | |
- as proto message in code: https://github.com/googleapis/google-cloud-python/blob/main/packages/google-cloud-retail/google/cloud/retail_v2/types/product.py | |
The model so far only supports product types "variant" and "primary" - which is a group of variants e.g. one productId with many skuIds. | |
Collections are currently not supported! | |
After validation product data has to be dumped `by_alias`: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding: utf-8 -*- | |
# Copyright (c) 2018-2022, earthobservations developers. | |
# Distributed under the MIT License. See LICENSE for more info. | |
import logging | |
import numpy as np | |
import pandas as pd | |
import utm | |
from scipy import interpolate |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
elif target.startswith("influxdb://"): | |
""" | |
====================== | |
InfluxDB database sink | |
====================== | |
Install Python driver:: | |
pip install influxdb |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Investigate problems when scanning huge directory | |
tree of DWD CDC HTTP server. | |
This repro will show the actual files as found in the url | |
python list-remote-files-dwd.py | grep -v zip | |
""" | |
import os | |
import requests | |
from typing import List |