Toad World® Forums

Use Toad Data Point - Query Local csv or Txt file

I have a large 5GB txt file that I cannot open in Notepad or Notepad++ or Excel. Is it possible to use TDP to query/SQL this file on my local C: Drive?

Could depend on how "beefy" your workstation configuration is (e.g. best if you have lots of RAM, etc....limited or low RAM could impact how big a file Toad DataPoint can read in).

However, would be easy to check... have Toad DataPoint's flat file connector (tab-delimited? fixed column?) point to the file's location and see what happens...

Thank you, I’ll give it a try!

image001.png

Let me know how you fare...

Perhaps another solution is to divide the source file into sizes your computer can handle. I have done this with a CSV file and then processed the 2 parts separately. This works if you do not have to share data thru the entire file.

I think there is a way in Toad to limit the number of lines you can read but I am not sure if you can start this process down into the file to begin a 2nd 1/2 read etc..

Hi Gary,

I don’t see a connection called ‘Flat File Connector’. This is what I see:

(I tried ‘csv’, but got an error)

Error when opening a .csv:

image004.png

image005.png

image001.png

Ah, yes... devil's in the details... so in order for Toad Data Point to properly access MS Office file types, including CSV and other flat files, which can be often affiliated with MS types, you'll need to make sure that you install the MS Access Database Engine package that Microsoft provides. This is also documented in TDP's Help content, including a link to the MS ADE download page.

You'll want to make sure that the bitness of TDP and the ADE match.

I ended up going another route….

I installed MS SQL Express 2017- free version on my local PC. I managed to find a video of how to tweak it so you can use MS SQL 2014 to open a file locally. So far, it’s working great!

Thank you for your suggestions…. !
:blush:

image001.png

Creative! I'll have to google the details!

Here’s the link….

image001.png