Join does not work as expected

Post Reply
flp
Posts: 51
Joined: Tue Sep 02, 2014 12:54 am

Join does not work as expected

Post by flp » Mon Nov 10, 2014 4:38 am

Files are uploaded here (also join.jpg):
https://www.dropbox.com/sh/q8a269fq3437 ... Q9yka?dl=0

My actual problem is that my join operator joins much less data than I expect.
I verified the existence of the expected data by creating the test.qry query, where I have attached the result tuple counts.
As you can see in the join.jpg, the last value for the first happening is 11544422638730918 which is far away from the expected 11544412920654172 + 1500000000000 (red color circles). I can't really understand that because I know through the test.qry that there is much more data.

Furthermore, the coalesce also aggregates one further tuple, which I didn't expect to be aggregated.
That can be seen coalesce.jpg (blue color circle), where the 11547081946080155 is still aggregating although the join.jpg (blue color circle) shows that this timestamp shouldn't fulfill the predicate anymore.
Attachments
coalesce.JPG
coalesce/task2_2 output
coalesce.JPG (95.18 KiB) Viewed 515 times
test_join.JPG
test.qry aggregation output
test_join.JPG (35.88 KiB) Viewed 516 times

User avatar
Marco Grawunder
Posts: 272
Joined: Tue Jul 29, 2014 10:29 am
Location: Oldenburg, Germany
Contact:

Re: Join does not work as expected

Post by Marco Grawunder » Mon Nov 10, 2014 11:08 am

Without looking at the query in detail...

One important thing: The timstamps from the tuples attributes are not used for processing. Processing is only done with the timestamps that can be found in the metadata. If you want to update the metadata with the timestamps from the tuple attributes you can use the TIMESTAMP operator (but this could change the query semantics).

http://wiki.odysseus.offis.uni-oldenbur ... p+operator

Regarding COALESE: Could you please test with the new version of COALESE (could be a bug in the old one ...)

http://wiki.odysseus.offis.uni-oldenbur ... e+operator

flp
Posts: 51
Joined: Tue Sep 02, 2014 12:54 am

Re: Join does not work as expected

Post by flp » Tue Nov 11, 2014 3:33 am

Looks like the new Coalesce fixes the bug.

I would really appreciate it if you can look into the join.
For me it looks crazy because the data exists, but it's not joined somehow.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest