Results set not showing all distinct results in dropdown

Hello,

I am using Toad Data Point 5.4.0.350 (64 bit).

I often use the dropdown menu on each column in the results set to see the distinct values in the column and filter results from there. Recently this dropdown box has not been showing all the values in the column.

Here are two examples from a single data set with 186,264 rows.

toad forums

The first image is a set of nulls at the bottom of the dataset. I wanted to filter to only see these nulls, but there is no (blanks) option to filter on. I can still use the custom option and it filters correctly.

The next example is a column that should have returned six semesters, but only showed five initially, and four after I clicked on the column to sort it to find an example for this post. I thought my code was excluding Fall 2021 when I noticed the issue. As you can see in the screenshot, now Fall 2021 appears in the results, but Spring 2021 is excluded, and Spring 2020 has disappeared as well.

Any ideas? Maybe my dataset is too large for Toad to capture all available values? Am I missing something in the settings?

Apparently one of my coworkers has this issue as well. Excel can show these distinct values, but it would be a massive pain to have to export to Excel for every single query that I run and I would like to fix this issue within Toad Data Point.

Thanks in advance,
Scott

Welcome to the ToadWorld forum for TDP, Scott.

I can't reproduce this in 5.6. Can you upgrade to the latest version? Actually, the latest version is 5.7, if you can grab that and upgrade.

It's true that the Excel-style filter icon should show all the values. However, several things come to mind that, hopefully might help.

  • What happens when you use the bottom right hand corner grab handle (dotted triangle) to increase the filter list area? Do more values show? Even so, it's weird that you're not getting any scroll bar to navigate through the list, as in snap #1 below...
  • There could be some data grids within the product for which not all capabilities (like Excel-Style filtering) are enabled. In which grid are you trying to do the filtering?
  • OK, gotta ask this, but are you fetching all the rows? e.g. if you execute a query in the TDP Editor, by default it only retrieves 1,000 rows on first fetch. The filter drop-downs are affected by how many rows are returned in the grid... see snap #2
  • There is another way to filter the contents of the data grid, see snap #3, although not as nice as the drop down list of values, granted.
  • Lastly, how are you connecting to your data sources? ODBC connectivity may carry with it some limitations.

Hope one of the above help, but if you can, I would upgrade to the latest version 5.7.

Hi Gary,

Thanks a lot for the suggestions.

I know my office generally upgrades software for everyone at once, so updating Toad to 5.6 or 5.7 isn't a simple thing, but I will talk with my coworker who handles software implementation for our department. I think it would be good to upgrade anyway.

EDIT: I did download Toad Data Point 5.6 at the recommendation of my coworker and the issue seems to be persisting. Depending on how the dataset is sorted, any new data after a certain point in the dataset is not included in the populated sort filter.

What happens when you use the bottom right hand corner grab handle (dotted triangle) to increase the filter list area? Do more values show? Even so, it's weird that you're not getting any scroll bar to navigate through the list, as in snap #1 below...

The dotted triangle will expand the box, but no more values show up. I do get a scroll bar for larger results set, like looking at individual IDs, but I thought this was weird because it was excluding values where the number of returned values was very small, >10.

There could be some data grids within the product for which not all capabilities (like Excel-Style filtering) are enabled. In which grid are you trying to do the filtering?

I did not know there were multiple data grids possible for Toad, I only thought there was the default grid that appears when you execute a query. I tried to search for data grids within Toad Data Point 5.7's documentation, but I didn't see anything. Do you have any links for further reading?

OK, gotta ask this, but are you fetching all the rows? e.g. if you execute a query in the TDP Editor, by default it only retrieves 1,000 rows on first fetch. The filter drop-downs are affected by how many rows are returned in the grid... see snap #2

I am getting all rows using the Fetch all option for this query. I think this issue has to do with how the results are returned, because if I order by student ID, then all six semesters show up, but if I order by semester, then only the first four semesters show up. If I order by student ID, all six semesters show up within the first few rows of the dataset.

If I order by semester, then the first four semesters occupy rows 1 through 122,608. The fifth and sixth semesters are all at row 122,609 or below, so it looks like Toad Data Point is returning all distinct values above a certain point, probably between 89,293 and 122,608 (where the fourth semester starts and ends).

I got the same results when I looked at the Academic_Standing column, where sorting it Descending gave nulls first, then Good Standing, Probation, and Warning, and excluded Dismissal (figure 1), which starts on line 183,472.

Sorting it Ascending gave the opposite results, excluding Probation and Warning (figure 2), which starts on line 173,362.

toad forums 2

So it looks like there's a cutoff point at which Toad no longer looks at the values to return results in the dropdown filter.

There is another way to filter the contents of the data grid, see snap #3, although not as nice as the drop down list of values, granted.

I don't see this filter icon on my version of Toad. Maybe it's located in a different spot, or it's available in an upgrade?

Lastly, how are you connecting to your data sources? ODBC connectivity may carry with it some limitations.

I am using an ODBC connection.

Thank you for the assistance so far!
Scott