Full Profiling on Hive generic ODBC result set not working since migration to 4.2

marcia.miskimen · April 17, 2018, 5:25pm

I recently migrated from TDP Pro 3.8 to TDP Pro 4.2 64-bit.

I find that when I have query results from a Hive query, and I click the Profiling tab, I see the summary data, but when I click the “Full Profiling” link on top-right, nothing happens.

Full profiling works OK in 4.2 when I had results from an Oracle query, however.

This worked perfectly fine in 3.8, for Hive queries (32-bit driver), I did it a lot. But now 4.2, Hive result sets are not sending to full profiling. ?? any advice?

Best I can determine, I get this exception, when I click the link:

New exception: 4/17/2018 7:17:42 PM

Quest.Toad.Exceptions.IgnoredException: Could not resolve connection for TRL: odbc://1A+Hive+IT+KERB+BW3/

Stack trace:

at Quest.Toad.Trl.TrlScheme.ResolveConnection(String trl, String xml, Boolean allowOpen, Boolean visible)
at Quest.Toad.Profiling.ToadProfileSource.GetConnection()
at Quest.Toad.Profiling.FullProfileOptionForm.set_ProfileSource(ToadProfileSource value)
at Quest.Toad.Profiling.FullProfileOptionForm.ProfileData(FastTable table, ToadDataAdapter adapter)
at Quest.Toad.Profiling.ProfileOverviewControl.fullProfileItem_ItemClick(Object sender, ItemClickEventArgs e)
at DevExpress.XtraBars.BarItem.OnClick(BarItemLink link)
at DevExpress.XtraBars.BarButtonItem.OnClick(BarItemLink link)
at DevExpress.XtraBars.BarItemLink.OnLinkClick()
at DevExpress.XtraBars.BarButtonItemLink.OnLinkAction(BarLinkAction action, Object actionArgs)
at DevExpress.XtraBars.ViewInfo.BarSelectionInfo.UnPressLink(BarItemLink link)
at DevExpress.XtraBars.Controls.CustomLinksControl.OnMouseUp(MouseEventArgs e)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at DevExpress.XtraBars.Controls.DockedBarControl.WndProc(Message& msg)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Thanks

AlekseyBazhenov · April 18, 2018, 12:36pm

Hi,

How did you connect to HIVE in TDP 3.8? Did you use ODBC connection?

To use the same ODBC 32 bit HIVE driver you will have to install 32 bit TDP or install 64 bit ODBC HIVE driver.

Bitnesses of TDP and ODBC driver have to be the same.

Regards

Aleksey

marcia.miskimen · April 19, 2018, 7:20am

Yes, connect to Hive in 3.8 and 4.2 using ODBC. 3.8 is 32-bit, 4.2 is 64-bit.

We use the “generic ODBC” category, and then point to the ODBC driver/config as set up in ODBC admin.

I can query Hive normally in 4.2 64 bit, that is not the problem.

The problem is when I use Profiling tool from a query results list.

Test#1

steps:

launch TDP 4.2 64 bit
connect to Hive , using ODBC 64 bit driver (Hortonworks ODBC driver)
Run query
Examine results
select Profiling tab in the results grid
Observe the summary profiling information -this works ok
Click “full profiling” at top-right -

Expected result : I should get the popup about full profiling options

Actual result: Nothing happens in the UI (but i see the exception in log, as in original post)

Test#2:

launch TDP 4.2 64 bit
connect to Oracle DB using Oracle driver
Run query
Examine results
select Profiling tab in the results grid
Observe the summary profiling information -this works ok
Click “full profiling” at top-right -

Expected result : I should get the popup about full profiling options

Actual result: popup appears and I can select Full profiling

Observe the full profiling data

AlekseyBazhenov · April 19, 2018, 10:13am

I see.

Could you try the latest TDP 4.3?

Thanks

Aleksey

marcia.miskimen · April 19, 2018, 2:58pm

Not easily, no. I have a convoluted approval process to get new software, it took me months just to get 4.2.

Are you confirming this is a known issue and is resolved in 4.3?

AlekseyBazhenov · April 20, 2018, 9:53am

Hi,

We reproduced the issue and created QAT-12819 to track it.

As a temporary workaround open data directory (Help|About and press Application Data Directory link), create a copy of KnowException.xml file and empty the original file.

Regards

Aleksey

marcia.miskimen · April 23, 2018, 11:48am

Hi Thank you for logging the issue.

I have some additional information discovered today.

I have several Hive connections, the majority are using Kerberos authentication, but one does not.

Only the Hive ODBC Kerberos connections are having the Profiling problem. The non-kerberos one works OK.

(Doesn’t really help me because the data I wanted to profile is only in the Kerberos db)

Secondly, I tried the fiddling with the KnownExecption file as you suggest, but it only got re-generated when I restarted Toad.

I also try deleted it while Toad running, and nothing changed for the Profiling behavior- still got same exception.

Do you have any ETA for a fix for QAT-12819? That is, what release it will be built to?

thanks

AlekseyBazhenov · April 23, 2018, 3:39pm

Thank you for additional info.

Right now I have no information about the time frame when the issue will be fixed but to push it a little bit I’d suggest to open a support case.

Regards

Aleksey

AlekseyBazhenov · October 26, 2018, 10:08pm

Hi

Could you please provide us the version of Hortonwork ODBC drive you use?

Thank you
Aleksey

marcia.miskimen · October 29, 2018, 1:45pm

i have Hortonworks ODBC Driver 2.1.2 , using 64-bit version, with Kerberos authentication to Hive.

can get here:
https://hortonworks.com/downloads/
then click on Add-Ons , on left, and should be able to find 2.1.2.

Debbie_Peabody · October 31, 2018, 7:15pm

Okay, we got the new driver and connection working. But we have tried everything we can and cannot reproduce. If you change the order of our connections or the Names of the connections does this get past the error? It seems some kind of lookup error.

marcia.miskimen · October 31, 2018, 10:12pm

I moved a connection, still same error.
I renamed the connection (just the Toad name) and still same error.

To be clear, the problem occurs ONLY when the connection is to a Kerberos Hive server.
I have another Hive that is NOT Kerberos, and profiling works ok there. CORRECTION: I Just tried this again after I posted this reply, and now the profiling isn’t working for the non-Kerberos Hive either. Very weird.

Does the profiling need to connect to the target again, even when it has all the data retrieved already?
That’s what puzzles me, why it needs to re-resolve the name.
Could it be that some of the Kerberos parameters are confusing the parser somehow? Just guessing.

I could try adding a completely new connection again - including a new ODBC conn, then point to it from a new Toad conn, and see what happens. UPDATE: Created all fresh, including a new connections.xml file, but still get same exception.

Can you confirm you are testing against Kerberos Hive?

There are a zillion parms for Kerberos and ODBC, so maybe we have to dig into those variants as well.

Cindy.Sang · November 5, 2018, 3:00am

Hi Marcia,

I could reproduce your issue with TDP 4.2 and 4.3 no matter kerberos or not, as long as it's ODBC way.
I did thoroughly testing on HIVE against various TDP versions, all of them work good with 5.0, could you try latest 5.0 beta? download from here:

https://forums.toadworld.com/t/toad-data-point-5-0-beta-program/41277

	TDP 4.2	TDP 4.3	TDP 5.0
Kerberos direct connection	works	works	works
Non Kerberos direct connection	works	works	works
ODBC Kerberos(v2.1)	has issue	has issue	works
ODBC Non Kerberos(v2.1)	has issue	has issue	works

Thanks,
Cindy

marcia.miskimen · February 12, 2019, 4:33pm

Haven’t checked in on this issue in a while.
My company is not supportive of me doing beta, so i have to wait for 5.0.
Is there any ETA on when 5.0 will be released?
I saw in another post there was a Feb1 target, but that has come and gone, and i did not see any announcements about 5.0 GA.
Please advise, thx.

Debbie_Peabody · February 13, 2019, 2:57am

New ETA is Mid April. If you email me directly I can get you a preview copy. debbie.peabody@quest.com But when I look at this issue QAT-12819, we never were able to reproduce so we did not make a change.

marcia.miskimen · February 13, 2019, 2:50pm

Hmm. that’s not what Cindy.Sang said above in earlier comment in this thread.
Looked like she reproduced it and confirmed the behavior in 4.2, 4.3, and 5.0.

can you find out for real?
It is a huge and slow process for me to get new versions of sfw (not allowed to download sfw from the outside, without jumping through many many hoops)

we are hoping to get approval for new version of TDP, but if that bug still exists, i’d like to know. It may affect our plans for upgrades.

Cindy.Sang · February 15, 2019, 5:49am

Hi,

Sorry to make you confused, I could reproduce this issue with TDP 4.2 and 4.3, but TDP 5.0 somehow works(it might be gone with control update), that’s why Debbie comment we did not make a code change, would you like to get a preview copy to take a try?

Cindy

marcia.miskimen · February 19, 2019, 4:11pm

ohhhh! ok, i see. yeah, that makes sense.

I looked into it , and i can get the preview copy. How can we make that happen?