Is there a limitation to the JSON output of a 'Get Items' call in Sharepoint?

[dynamicsgeek]

I have a flow that has been working fine for about a month.

It compares items on a SharePoint list to another list of items, and if there is no match, it creates the item on the list.

It stopped working because it looks like the number of rows returned is incomplete. Searching this forum, most posts refer to a 5,000 item limit.

My sharepoint list has only 150 items, so I don't think this is the problem.

When I inspect the output, the document contains about 5,419 lines (not items) of JSON. And some of my data is not there.

I've tried experimenting with the different options in Get Items, such as "Limit Entries" to folder and "Include nested items".

These tend to alter the number of lines output, but don't contain the data I'm looking for.

There was also mention of a column limit, so I tried creating a view with less columns, this did reduce the amount of data - but my data is not there. As a side note, I cannot get certain columns to be removed from the JSON, even if I remove them from the view, and they are not built-in columns, but columns I added, is this working as expected?

If I use an ODATA filter, such as items added after a certain date, for example - InactiveDate ge '@{addDays(utcNow(),-16)}'

which returns the most recent 16 days of data, I do see the entries I'm looking for.

But since I am not near the 5,000 item limit, why did it not return all items?

why do I have to filter using OData to get the data I am looking for?

Shouldn't this return all items less than 5,000 items with these options? Is there another limitation in file size or in output that I am missing?

[dynamicsgeek · Accepted solution]

I'm not sure why this didn't come up in searches on this site (I wasn't using the right terms), but this fixed my issue https://www.youtube.com/watch?v=zuD-OQYE1AI

Once I started looking searching for 'pagination' instead of 'get items limit' this started to appear.

This resolved my major issue, but I'm still confused on why using a view didn't limit my columns in the JSON output.