Toad World® Forums

Is it possible to load big files via TDP

Hello,

I was trying to find a way to load a file with about 1 mln records via TDP. I used Oracle imp utility but was wondering if Toad has this kind of option. In the past I tried to load 300 K records but the UI crashed.

Thanks,
Chris

I use the latest version 3.7 Pro Edition on Windows 7

TDP is able to handle large file import. An important factor here is "processing row count", which is 500 by default. If your table has a lot of columns or some columns are very large, I would suggest that you change it to a smaller number like 100 or 50 to avoid possible out of memory issue which might lead to crashing, otherwise, if you only have a few columns, you can try a bigger number such as 2000 rows to maximize the performance. Please let us know if it helps or not.

In general when dealing with large files use CSV and not Excel as input. It is actually more efficient to load large data not from a file but from a query. You can query one database and import into another. Use the Add Query option and then follow the row count suggestion Kiki’s suggestions.

Thank you for reply. Actually it put on the right path. I completely forgot that I could use an import option and select a CSV file as a source. I was able to test 500 K records (52 attributes) without any problems. Next test it will be with a file containing 2mln records.

I will let you know the result.

If your file size is greater than 2 Gb you may have trouble. Less than that should be no problem. You can break the file up into smaller than 2 Gb pieces and do multiple imports if you need to.