Skip to content

Instantly share code, notes, and snippets.

@valoricDe
Last active April 2, 2023 17:37
Show Gist options
  • Save valoricDe/1ef621a38b2557d63f3c22b108b7f044 to your computer and use it in GitHub Desktop.
Save valoricDe/1ef621a38b2557d63f3c22b108b7f044 to your computer and use it in GitHub Desktop.
fraabe
async function getHtmlDocumentById(id: number | string) {
const baseUrl = 'https://www.whiskybase.com'
const response = await fetch(
`${baseUrl}/EditCustomer?id=${id}`,
{
credentials: 'omit',
headers: {
'User-Agent':
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:87.0) Gecko/20100101 Firefox/87.0',
Accept: '*/*',
'Accept-Language': 'en-US,en;q=0.5',
},
referrer: `${baseUrl}`,
method: 'GET',
mode: 'cors',
}
)
if (response.status >= 400) throw Error(await response.text())
const html = await response.text()
const {
window: { document },
} = new JSDOM(html)
return document
}
function extractData(document) {
const currentPrice = document.querySelector(
'.block-price p:last-child'
)?.textContent
}
function exportDatabase() {
const max = 5000;
for(let id = 0; id < max; id++) {
const doc = getHtmlDocumentById(id)
const data = extractData(doc)
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment