Hi!
We have a new feature - Analyze Join. Here is a blog post describing it.
Debbie
Hi Debbie,
I’m curious about this feature - I haven’t downloaded the 3.7 Beta yet to try it out, but how does it handle tables with very large datasets?
For example, one report we run daily has to join one table of 33 Million rows to another table of approximately 16 Million and find the matches between. There are some predicates in place that take advantage of the indexes, but does the Analyzer use those same indexes when analyzing the join? Or does it attempt to retrieve the entire data set from both sides?
The evaluation of this join will happen on the server executing a relatively ugly looking piece of SQL that should none the less execute relatively efficiently assuming there are indexes in place for the columns needed.
However if there are no indexes on the columns being tested you are in for a long wait I’m afraid.
That said analyzing a 33 million row by 16 million rows analyze will take a while (Just doing one full table scan would take a while on those kind of tables and this will need to do an index lookup in the second table for every row in the first table).