Loop through List(Integer)

stefan
Posts: 85
Joined: Tue Jul 12, 2016 1:03 pm

Re: Loop through List(Integer)

Post by stefan » Wed Nov 30, 2016 8:12 pm

Hi Marco,

great overall it works fine and I think I can build my functionality with these functions. I noticed during testing one behavior that I did not expect:
In the MAP Operator I use this list function:

Code: Select all

search(departures[desiredDepartureWeekday],kvread("curStartTime"),true)
If curStartTime is 2200 and the departures list contains values from 600 till 2000 the above search returns 2000. Because of the last attribute (=true) the search should only return values >= 2200. But if the search could not find a value at all it returns the greatest one even if its smaller. Should it work like this? If yes, I just have to check if the value is greater that 0. I just wanted to inform you. :)

greetings,
Stefan

User avatar
Marco Grawunder
Posts: 272
Joined: Tue Jul 29, 2014 10:29 am
Location: Oldenburg, Germany
Contact:

Re: Loop through List(Integer)

Post by Marco Grawunder » Thu Dec 01, 2016 10:00 am

Yes, of course. You are right. Fixed in the next build.

stefan
Posts: 85
Joined: Tue Jul 12, 2016 1:03 pm

Re: Loop through List(Integer)

Post by stefan » Fri Dec 02, 2016 9:20 pm

Hi Marco,

great, now it is working correctly. Did you changed something in the execution order of the MAP operator or the kvread/kvwrite? I already tested my development and it seems to be fine. But since my last pull I get strange results. It seems that the order of read/write is different sometimes. In some cases old values are read that already should be replaced, sometimes it even seems that the calculation is done twice.

I noticed that during some changes on my code. To ensure that I did not mess up something I used an old version of the code but I have still strange results. So I started testing. If I start the query with the same input I get different outputs. I always dropped all sources, sinks and queries to ensure that there are no side effects. Even if an erros is in my code the output should always be the same, shouldnt it?

greetings,
Stefan

User avatar
Marco Grawunder
Posts: 272
Joined: Tue Jul 29, 2014 10:29 am
Location: Oldenburg, Germany
Contact:

Re: Loop through List(Integer)

Post by Marco Grawunder » Mon Dec 05, 2016 9:47 am

Hmm.

No. I did not change anything in the execution order.

If the input is the same and all processing is determinstic there should be always the same ouput. Some strange behaviour could occur when you using element based windows and there are too many equals timestamps. Can you provide us the whole queries?

Can you create an output directly before the MAP Operator and after the MAP-Operator (you can attach a second CSVFileSink to each operator)?

Greetings,

Marco

stefan
Posts: 85
Joined: Tue Jul 12, 2016 1:03 pm

Re: Loop through List(Integer)

Post by stefan » Mon Dec 05, 2016 6:56 pm

I tried to reproduce this behavoir today but now everything is working as it is supposed to be. I used the same code, the same input etc. I just pulled the latest sources and did a manual clean of the project. Maybe there happend something that is fixed by the cleaning and recompiling. Really strange. But as long as it is working now its fine for me.

In my MAP operator (see the full query I sent you today) I use an temporary attribute for a calculation step:

Code: Select all

['kvwrite("curArrivalTime", toInteger(((kvread("curArrivalTimeInMin") - kvread("curArrivalTimeInMin") % 60)/60)*100 + (kvread("curArrivalTimeInMin") % 60)))','tmp_calcArrivalTime'],
If I open the current (query) plan and click "Show Stream Elements" of that MAP operator I noticed these two things:
1. The header names of the attributes are not in the correct order. That means that e. g. the values of the "departures" attribute are called "travelTime" etc. I think there goes something wrong because I use "keepinput" and "removevalues".

2. I get an error in the console during execution:

Code: Select all

166215 ERROR StreamTableEditor  - Could not retrieve attributeValue - de.uniol.inf.is.odysseus.rcp.viewer.stream.table.StreamTableEditor$8.update(StreamTableEditor.java:381) 
java.lang.ArrayIndexOutOfBoundsException: 42
	at de.uniol.inf.is.odysseus.core.collection.Tuple.getAttribute(Tuple.java:286)
	at de.uniol.inf.is.odysseus.rcp.viewer.stream.table.StreamTableEditor$8.update(StreamTableEditor.java:365)
	at org.eclipse.jface.viewers.ViewerColumn.refresh(ViewerColumn.java:154)
	at org.eclipse.jface.viewers.AbstractTableViewer.doUpdateItem(AbstractTableViewer.java:385)
	at org.eclipse.jface.viewers.StructuredViewer$UpdateItemSafeRunnable.run(StructuredViewer.java:473)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
	at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:50)
	at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:173)
	at org.eclipse.jface.viewers.StructuredViewer.updateItem(StructuredViewer.java:2176)
	at org.eclipse.jface.viewers.AbstractTableViewer.createItem(AbstractTableViewer.java:276)
	at org.eclipse.jface.viewers.AbstractTableViewer.internalRefreshAll(AbstractTableViewer.java:723)
	at org.eclipse.jface.viewers.AbstractTableViewer.internalRefresh(AbstractTableViewer.java:615)
	at org.eclipse.jface.viewers.AbstractTableViewer.internalRefresh(AbstractTableViewer.java:607)
	at org.eclipse.jface.viewers.StructuredViewer$7.run(StructuredViewer.java:1526)
	at org.eclipse.jface.viewers.StructuredViewer.preservingSelection(StructuredViewer.java:1462)
	at org.eclipse.jface.viewers.StructuredViewer.preservingSelection(StructuredViewer.java:1423)
	at org.eclipse.jface.viewers.StructuredViewer.refresh(StructuredViewer.java:1523)
	at org.eclipse.jface.viewers.ColumnViewer.refresh(ColumnViewer.java:522)
	at org.eclipse.jface.viewers.StructuredViewer.refresh(StructuredViewer.java:1482)
	at de.uniol.inf.is.odysseus.rcp.viewer.stream.table.StreamTableEditor$1.run(StreamTableEditor.java:148)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:135)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4155)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3772)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$4.run(PartRenderingEngine.java:1127)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:337)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1018)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:156)
	at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:654)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:337)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:598)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
	at de.uniol.inf.is.odysseus.product.studio.starter.OdysseusStudioApplication.start(OdysseusStudioApplication.java:68)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:669)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:608)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1515)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1488)
BOTH is no problem for me. The calculation uses the correct values (so just the displayed stream table shows the wrong header) and the calculation of "tmp_calcArrivalTime" is done as it is supposed to be. I just noticed it and want to inform you. :)

greetings,
Stefan

User avatar
Marco Grawunder
Posts: 272
Joined: Tue Jul 29, 2014 10:29 am
Location: Oldenburg, Germany
Contact:

Re: Loop through List(Integer)

Post by Marco Grawunder » Tue Dec 06, 2016 8:18 am

Yes, this refresh/clean is a know problem of eclipse and osgi. Additionally, we always start eclipse with -clean option.

Thank you for your feedback! I will have a look at the other error. I guess there goes something wrong with the output schema calculation.

Greetings,

Marco

User avatar
Marco Grawunder
Posts: 272
Joined: Tue Jul 29, 2014 10:29 am
Location: Oldenburg, Germany
Contact:

Re: Loop through List(Integer)

Post by Marco Grawunder » Wed Dec 07, 2016 11:57 am

Ok. This is no error on the schema. This seems to be fine. So there seems to be another problem.

Could you please create a bug report and attach some output of the MAP-Operator in textual format (Show stream elemens - List)?

Greetings,

Marco

stefan
Posts: 85
Joined: Tue Jul 12, 2016 1:03 pm

Re: Loop through List(Integer)

Post by stefan » Mon Dec 12, 2016 2:43 am

Hi,

I got a short question about lists again:
If I read the value of a List(Integer) from csv and it is represented as '[900]' in the csv file than Odysseus is able to work with this. If I have an empty list, represented in csv like '[]' it will be represented in Odysseus as [null].
If I get an empty list from my Python script (as JSON) a empty list is [].

My question: Is this [null] an error or the correct representation? Is there a way to replace the value [null] by []? I tried this expressions line by line:

Code: Select all

moSrcUuid = MAP({
		EXPRESSIONS = [
			///['eIf(departureMonday==[null],toList([]),departureMonday', 'departureMonday2'],
			///['eIf(isNull(departureMonday),toList([]),departureMonday', 'departureMonday2'],
			///['eIf(isEmpty(departureMonday),toList([]),departureMonday', 'departureMonday2'],
			///['eIf(isEmpty(departureMonday),0,1', 'departureMonday2'],
			['eIf(isNull(departureMonday),0,1', 'departureMonday2']
		],
		KEEPINPUT = true
	},
	moCsvSource
)
The creation of a new empty list always throughs an error, the isNull and isEmpty functions always return false. The input is always "[null]". How should this work? What can I try?

Thanks,
Stefan

User avatar
Marco Grawunder
Posts: 272
Joined: Tue Jul 29, 2014 10:29 am
Location: Oldenburg, Germany
Contact:

Re: Loop through List(Integer)

Post by Marco Grawunder » Mon Dec 12, 2016 7:04 pm

Ok. This this is a special case where only one value is inside the list and this value is null (I think, this can only happen in cases where List<List<>>). Will be fixed in the next build.

Greetings,

Marco

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest