Rapid Prototyping to split tuple values in multiple streams

User avatar
Marco Grawunder
Posts: 272
Joined: Tue Jul 29, 2014 10:29 am
Location: Oldenburg, Germany
Contact:

Re: Rapid Prototyping to split tuple values in multiple streams

Post by Marco Grawunder » Mon Nov 28, 2016 4:00 pm

Hmm, this does not really makes sense to me.

You can click on each operator in the query plan and print its result. There should be no difference for an attribute that is not touched.

What happens when you change the code as follows:

Code: Select all

/// Add a global unique identifier to Mobility Request CSV Data
inputWithUuid = MAP({
   expressions = [
      ['uuid()', 'mr_uuid'],
      'idInCSV'
   ]
}, csvInput
)

/// Prepare Neo4j related attributes for RabbitMQ
projectedInput = PROJECT(
   {
      attributes = ['mr_uuid', 'idInCSV']
   }, inputWithUuid
)

P.S:You can run a query twice if you click on the query in the query view. ...

Or do you use: Show data on the sources? Each time you call this, a new query is created and the function uuid() is called again

There is typically no need to use ":=" in most cases. Only when you want to reuse this in other queries.

stefan
Posts: 85
Joined: Tue Jul 12, 2016 1:03 pm

Re: Rapid Prototyping to split tuple values in multiple streams

Post by stefan » Tue Nov 29, 2016 2:37 pm

Ok, I am not able to test this at the moment, I will check that this afternoon or tomorrow.
Ah, ok, you mean by using start or restart of a query. Ok, yes, that way I know.
What I am doing: I create views by using := because it found it easier to track all the different steps. I just have to open it and I can see whats happening. But I want to change this for the final deployment.

So I start without any source, sink, query etc. I execute the query (full source code), and open the views by clicking "show data" on the sources for inputWithUuid and projectedInput. Than I can see what is the result for each operator. If I compare these results (manually), I can see the differences.

Are these differences caused by my testing behavior?

User avatar
Marco Grawunder
Posts: 272
Joined: Tue Jul 29, 2014 10:29 am
Location: Oldenburg, Germany
Contact:

Re: Rapid Prototyping to split tuple values in multiple streams

Post by Marco Grawunder » Tue Nov 29, 2016 3:24 pm

I think so. The input file is very short and when you look at the output of one source and after that to the output of another source (from the same query), it could be that the processing is already done.

In fact "Query source" (former "Show data") is the short cut for the installation of a query "out = source" and the presentation inside a tabular view. If you call it twice a mechanism called query sharing gets active, that recognizes "These operators are already used so I use them again."

stefan
Posts: 85
Joined: Tue Jul 12, 2016 1:03 pm

Re: Rapid Prototyping to split tuple values in multiple streams

Post by stefan » Tue Nov 29, 2016 5:27 pm

Hmm, ok. I added a time delay in the read csv query for that but I understand your point. Whats the way I should check this?

I will try to write the result of the two queries to a csv file to see whats happening. The reason why I noticed this was the following: I wanted to join two streams and got no results. I checked it and saw that there are no matching partner. But maybe there was another error that was resulting in an empty result...

Thanks,
Stefan

stefan
Posts: 85
Joined: Tue Jul 12, 2016 1:03 pm

Re: Rapid Prototyping to split tuple values in multiple streams

Post by stefan » Wed Nov 30, 2016 2:16 pm

Ok, I think I figured out the problem.
I used my example and executed the code in one "full" query by (without views)

Code: Select all

#RUNQUERY
csvInput = ACCESS({
...

inputWithUuid = MAP({
...

projectedInput = PROJECT(
...

output = SENDER({
...
            inputWithUuid
          )

output2 = SENDER({
...
            projectedInput
          )
... and I also executed the code by using views:

Code: Select all

#RUNQUERY
csvInput := ACCESS({
...

#RUNQUERY
inputWithUuid := MAP({
...

#RUNQUERY
projectedInput := PROJECT(
...

output = SENDER({
...
            inputWithUuid
          )

output2 = SENDER({
...
            projectedInput
          )
Notice that I use in the second part views and the "#RUNQUERY" command.
In the first example the output/the uuid is the same in each csv file. In the second example I get different uuids.

Thanks, this is good to know!

greetings,
Stefan

User avatar
Marco Grawunder
Posts: 272
Joined: Tue Jul 29, 2014 10:29 am
Location: Oldenburg, Germany
Contact:

Re: Rapid Prototyping to split tuple values in multiple streams

Post by Marco Grawunder » Wed Nov 30, 2016 2:27 pm

Good to hear.

Thank you, for your feedback :-)

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest