Get all data from JSON api w/ pagination enabled?

Programming
,
1

I need to access all 50 pages of data from the https://hn.algolia.com/api/v1/search_by_date?query=nodejs API. My current code retrieves only the first page (page 1) of data:

urlApi = 'https://hn.algolia.com/api/v1/search_by_date?query=nodejs'

let repo = await axios.get(urlApi);
repo.data.hits.map((data) => {
    console.log(data);
 });

However, the JSON response contains 'nbPages': 50 indicating that there are 50 pages of data. Do you have any ideas for retrieving all 50 pages of data?

2

You can retrieve all 50 pages of data by using a loop to make multiple requests to the API, incrementing the page parameter in the URL with each request. Here’s an example code snippet using async/await and axios:

const urlApi = 'https://hn.algolia.com/api/v1/search_by_date?query=nodejs&page=';

const fetchData = async () => {
  let allData = [];
  for (let i = 1; i <= 50; i++) {
    const response = await axios.get(urlApi + i);
    allData.push(...response.data.hits);
  }
  console.log(allData);
};

fetchData();

This code uses a for loop to iterate through the 50 pages of data, making a request to the API with each iteration and pushing the results into an array. The ... syntax is used to spread the array of hits from each response into the allData array, rather than pushing the array itself. Finally, the console.log statement outputs the combined array of all 50 pages of data.