Regression skip-list hackathon

[biagas]

In preparation for the skip-list hackathon, here is a table with the items in the skiplist, as well as associated ticket numbers that I could find. To save space I didn't list the run modes where the failures occur.

Platform	File	Cases	Ticket #s	Comments
all	databases/diff.py		#1729 #1911
all	databases/exodus.py	exodus_13a exodus_13b	#5082
all	databases/pixie.py	pixie_04	#3090 #3148
all	databases/silo.py	silo_curvilinear_3d_surface_3	#17708	The test passes, but it looks like it is missing mesh lines in one direction in the 2d surface in 3d.
all	databases/zipwrapper.py		#5090
all	hybrid/ clonecopy.py		#5493
all	hybrid/ddf_vs_dbinning.py		#2835	Cyrus This is still an issue, but I recommend we only skip the failing test not the entire file see: #18683
all	hybrid/missingdata	missingdata_0_04	none	Cyrus -- missingdata_0_04 fails for ice t case. This is a text test with a pick result. In ice t the pick fails to find anything, other cases it works.
all	hybrid/movie.py		#18561
all	hybrid/ selections.py		none	A few can be rebaselined, but most of the images show the selections not working properly.
all	hybrid/selections_pp.py		none	It is trying to open the file /Projects/VisIt/trunk/build/data/cq_pp_test_data/unsorted.h5part. This clearly doesn't exist. Couldn't find the file in our testdata. Should remove if we can't find it.
all	operators/tranform.py	ops_transform05	#270
all	queries/IntegralCurveInfo.py		none	Cyrus - the current values are different than the baselines. Running produces more output, some of the output looks similar, but there are eventually lots of diffs. Maybe re-baseline? serial and parallel look like the same results (quick inspection)
all	rendering/compositing.py		#266	justin - this is some kind of transparency issue again. Seems similar to #022862 Should remain on the skip list for now.
all	rendering/line3d.py	line3d07 - line3d12	none	these fail for the ice-t, pointing to a real issue with icet .
all	rendering/ospray.py		none	Creates 2 image, the first clearly has shadows and looks correct. The second looks fine, but I can't verify if it is using ospray, I assume so. We should probably remove from skip list.
all	rendering/pixeldata.py	pixeldata_0_05 pixeldata_0_07 pixeldata_0_09 pixeldata_0_11	#18685	justin - these are failing and I opened a ticket about it
all	session/correlationsession.py	correlationsession00		justin - this is succeeding now; we can remove it from the skip list
all	session/legendproperties.py	legendproperties00 legendproperties01	#350	justin - this is succeeding now; we can remove it from the skip list. The associated ticket should be closed after we change the test to turn screen capture ON.
all	simulation/curve.py	curve03 curve04	none	based on commit message they fail too frequently - justin - we should move away from baselines for these tests and move towards the approach @markcmiller86 is using where we make sure that no errors are thrown and consider that a pass.
all	simulation/life.py	life03	#5077	based on commit message they fail too frequently
all	simulation/zerocopy.py	zerocopy02	none	based on commit message they fail too frequently
all	quickrecipes/how_to_start.py		none	a visit.session file needed for the test was not committed with original work. See below for a comment where this file is attached. The second issue about referencing a person's home directory is actually a bit of a misread of the code. It does reference ~juanita/silo/stuff/wave.visit as the silo data file. But, that is because this code is verbatim included in our docs and I didn't want test harness paths appearing there. So, there is logic that maps these strings to silo_data_path() arguments. On pascal, this works fine when I run it manually.
all	quickrecipes/ opening_compute_engine.py		none
win	databases/ffp.py	ffp_03		no libstripack on windows (requires fortran compiler), can it be skipped via logic in the test instead of being in skip list
win	databases/silo.py	silo_26 silo_36 silo_37 silo_38 silo_39	#18682 (silo_26) #18684 (silo_36-silo_39)	silo_26 is testing largefile.silo and already has skip logic in the file. I think the logic can be modified to locate the file on Windows. silo_36 - silo_39 testing wave_1file.visit and cannot read the individual files listed in the .visit file.
win	databases/tecplot.py	tecplot_27		intermittent random failure, sometimes the triangle is red, sometimes blue
win	hybrid/field_operators.py	field_op_04		This is passing and should be removed from the skip list.
win	plots/contour.py	contour_lineStyle01		Should be removed, test no longer exists
win	plots/label.py	label_8_02		this is passing with current windows baselines and could be removed from skip list. However the results don't look right in many ways, but the same is true for the linux results. Probably related to #3103
win	plugins/*.py			needs extra wiring to work on Windows
win	queries/flatten.py	multi_rect2d_box_nodes multi_rect2d_box_zones	#17767
win	simulation/batch.py simulation/updateplots_session.py		#18651	Not applicable to Windows as they attempt to run in batch.
win	simulations: curve.py domainlist.py globalids.py life.py mandelbrot.py ucdcurve.py updatplots.py zerocopy.py		#18652	These use the 'step' command that doesn't play well with the testing infrastructure on Windows, and hang.
win	unit/launcher.py		#18651	Not applicable on Windows.

[biagas]

quickrecipes/saving_images test should be in the skip list for Windows, as it causes a parallel engine crash, but I didn't know how to add it to the skip list (uses TestValueEQ, TestPOA, TesFOA). The ticket is #18491.

[markcmiller86]

Here are skips via commenting out...

./meshtype/csg.py:#Test("csg_06")
./databases/leos.py:#Test("leos_02")
./databases/tecplot.py:#Test("tecplot_07")
./databases/ProteinDataBank.py:#    Test(testname, swa)
./faulttolerance/badfile.py:#TestText("badfile_01", e)
./queries/scf.py:#TestText("scf_04", t)
./queries/scf.py:#TestText("scf_06", t)
./rendering/scalable.py:#TestSection("Stereo Modes")
./rendering/scalable.py:#Test("scalable_11")
./plots/curve.py:    #Test("curve_2_04")
./hybrid/selections.py:#    TestText("selections_4_03", str(GetSelectionSummary(selName)))

[biagas]

I'm looking at csg.py

[biagas]

csg_06 yields a good image on Windows and linux and it looks similar to csg_07, so I don't know why the code was commented out. I looked at the blame on the file, and it was commented out 17 years ago, so something was probably fixed since then. It should be uncommented and a baseline created.

[markcmiller86]

Attaching here the visit.session file needed for how_to_start.py for the using session files test.
visit.session.gz

[biagas]

leos.py's own comments say the commented out test was obsoleted, so perhaps it should just be removed:

visit/src/test/tests/databases/leos.py

Lines 39 to 44 in 2464615

	# Test auto full frame. This test was obsoleted when
	# auto ff became the default
	#
	#v.fullFrameActivationMode = v.Auto
	#SetView2D(v)
	#Test("leos_02")

[biagas]

tecplot.py lists a reason for its test being skipped.

I think the referenced ticket was a redmine issue and I couldn't find anything related on github.
I tried running the test and the results don't look right so I think it is still a problem.

visit/src/test/tests/databases/tecplot.py

Lines 109 to 121 in 2464615

	# We must skip this one, as it has column-major
	# ordering instead of row-major, and we are
	# expecting row-major. There is nothing in the
	# tecplot file format to specify which type it is,
	# so this must wait until we have options for
	# the database plugins. See '2440.
	##OpenDatabase(data_path("tecplot_test_data/cylindrical.tec"))
	#AddPlot("Mesh","mesh")
	#DrawPlots()
	#SetTheView()
	#Test("tecplot_07")
	#DeleteAllPlots();

[biagas]

ProteinDatabank.py: the comment in Mark's list above is for using larger images, not skipping a particular test.
However, there is testing being skipped in the file because it does not work (according to its own comments and my allowing it to run to see what happens).
The skipped test section (test4) is trying ReplaceDatabase.

I couldn't find a related ticket, so one should be created.

[biagas]

scf.py indicates an issue with Localized Compactness Factor that needed to be sorted out.
I ran the test in serial and got a result, but I don't know if it is correct. Couldn't find a related ticket.

visit/src/test/tests/queries/scf.py

Lines 122 to 125 in 2464615

	# Don't test localized compactness factor until radii is sorted out.
	#Query("Localized Compactness Factor")
	#t = GetQueryOutputString()
	#TestText("scf_04", t)