Skip to content

Instantly share code, notes, and snippets.

@iampavangandhi
Created October 5, 2020 05:17
Show Gist options
  • Save iampavangandhi/f3c87b4ac4360b18794c7578507dc829 to your computer and use it in GitHub Desktop.
Save iampavangandhi/f3c87b4ac4360b18794c7578507dc829 to your computer and use it in GitHub Desktop.
Example to scrape posts and put into a CSV file using Nodejs & Cheerio.
const request = require('request');
const cheerio = require('cheerio');
const fs = require('fs');
const writeStream = fs.createWriteStream('post.csv');
// Write Headers
writeStream.write(`Title,Link,Date \n`);
request('http://codedemos.com/sampleblog', (error, response, html) => {
if (!error && response.statusCode == 200) {
const $ = cheerio.load(html);
$('.post-preview').each((i, el) => {
const title = $(el)
.find('.post-title')
.text()
.replace(/\s\s+/g, '');
const link = $(el)
.find('a')
.attr('href');
const date = $(el)
.find('.post-date')
.text()
.replace(/,/, '');
// Write Row To CSV
writeStream.write(`${title}, ${link}, ${date} \n`);
});
console.log('Scraping Done...');
}
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment