Page 2 of 2

Re: Working with JSON

Posted: Fri Sep 19, 2014 7:58 am
by Marco Grawunder
I think the problem could be, that the reading of the elements is too fast. We use timestamps to simulate element based windows, but when den elements have the same timestamps there is no way to distinguish them. In your case, timestamps are based on system time and the granularity is milliseconds.

Could you please add in the source definition the option ['scheduler.delay','1'] and test again?

You shoud now get something like:

Code: Select all

1|10|10|10.0 | META | 1411113161849|1411113161866
2|11|10|21.0 | META | 1411113161866|1411113161881
3|12|10|33.0 | META | 1411113161881|1411113161897
4|13|10|46.0 | META | 1411113161897|1411113161913
5|14|10|60.0 | META | 1411113161913|1411113161928
6|15|10|75.0 | META | 1411113161928|1411113161944
7|16|10|91.0 | META | 1411113161944|1411113161960
8|17|10|108.0 | META | 1411113161960|1411113161975
9|18|10|126.0 | META | 1411113161975|1411113161991
10|19|10|145.0 | META | 1411113161991|1411113162006
10|20|11|155.0 | META | 1411113162006|1411113162007

This is because of the sliding window. For each new element the window is moved one position. So in the first window, there is only the first element, then the first and the second and so on. Finally, the last two elements are from a full window of ten elements.

If you want distinct windows, you can use the SLIDE-Parameter, e.g. SLIDE = 10.

If you only want windows with 10 elements you could e.g. use a selection on the count attribute after the aggregation.

Re: Working with JSON

Posted: Fri Sep 19, 2014 2:13 pm
by harpreet
Thank You so much. It worked.

I'll trouble you guys again if I get stuck again. Hope you don't mind.


Thanks,
Harpreet

Re: Working with JSON

Posted: Fri Sep 19, 2014 2:39 pm
by Marco Grawunder
Thank You so much. It worked.
That's fine :-)
I'll trouble you guys again if I get stuck again. Hope you don't mind
Hey, thats whats this forum is for ;-)

Re: Working with JSON

Posted: Sat Mar 11, 2017 12:41 am
by daniela_4444
Hi everyone

I am new here and I would like to create a JSON stream source in Odysseus. Unfortunately I always receive the following error:

'Parsing and Executing Query' has encountered a problem.

Script Execution Error: No such data handler: KeyValueObject
No such data handler: KeyValueObject


Does anyone have an idea why I receive this error?

Thank you in advance.

Best regards,
Daniela

Re: Working with JSON

Posted: Sat Mar 11, 2017 9:20 am
by Marco Grawunder
Hi,

how do you use Odysseus? Are you using it as the downloaded product? Then you will need to install the KeyValue Feature

https://wiki.odysseus.informatik.uni-ol ... w+features

When you are using Odysseus from source code, you need to add the feature to the used product and call synchronize.

Greetings,

Marco

Re: Working with JSON

Posted: Fri Mar 31, 2017 9:18 am
by daniela_4444
Hi Marco

thanks a lot, it is now working. Now I tried to use a JSON file as source but unfortunately my PQL/CQL queries do not work. I always receive a null pointer exception. Where do I have to store the JSON file or what may be the problem in my case? I already tried the example from your last posts.

My query looks as follows:
#PARSER PQL
#RUNQUERY
///Your first pql-query here
mysource := ACCESS({
source='sourceName',
wrapper='GenericPull',
transport='File',
protocol='JSON',
dataHandler='KeyValueObject',
options=[['filename','C:/Users/Daniela/workspace/myproject/sample.json']]})

My example JSON file looks as follows:
{"person":[
{"firstName":"a", "age":"27"},
{"firstName":"b", "age":"28"},
{"firstName":"c", "age":"24"},
{"firstName":"d", "age":"25"},
{"firstName":"e", "age":"10"},
{"firstName":"f", "age":"56"},
{"firstName":"g", "age":"45"},
{"firstName":"h", "age":"36"},
{"firstName":"i", "age":"31"},
{"firstName":"j", "age":"35"},
{"firstName":"x", "age":"25"}]}

Thank you in advance!

Best regards,
Daniela

Re: Working with JSON

Posted: Fri Mar 31, 2017 9:36 am
by Marco Grawunder
Hi Daniela,

instead of

options=[['filename','C:/Users/Daniela/workspace/myproject/sample.json']]})

you can use

options=[['filename','${PROJECPATH}/sample.json']]})

What is the null pointer message?

Re: Working with JSON

Posted: Sat Apr 01, 2017 12:46 pm
by daniela_4444
Hi Marco

thank you for your message. Unfortunately it still does not work. I always receive a java.lang.NullPointerException. I also tried to use a csv file as source. It seems to work but I do net see it in my source list. It is always empty. Do you have an idea what could be the problem?

Thank you in advance.

Regards,
Daniela

This is my code for the csv source:
#PARSER PQL
#RUNQUERY
///Your first pql-query here
input = ACCESS({
source='test',
wrapper='GenericPull',
transport='File',
protocol='CSV',
datahandler='Tuple',
options=[
['delimiter',','],
['textDelimiter',"'"],
['readfirstline','true'],
['delay','100'],
['filename', '${PROJECTPATH}/test.csv']
],
schema=[
['firstName','String'],
['age','Integer']
]
}
)

Re: Working with JSON

Posted: Sat Apr 01, 2017 3:31 pm
by Marco Grawunder
Are you using the latest Odysseus version? If not, please update.

Else: Could you please make a bug report?

https://wiki.odysseus.offis.uni-oldenbu ... port+a+bug

Greetings,

Marco