Auto-load data and 'scene' (or state) on launch

For me, a main motivation of using visualizer is to be able to share simulation results with a collaborator without them needing the hardware or software to open the data. Because of this we need to assume that the collaborator is also not familiar with ParaView.

What I'd like to do is preload a dataset in a saved state (e.g. using .pvsm). I've found documentation for visualizer regarding installation but not regarding usage. Could you please point me in the right direction?

Also, does visualizer accept all of the same input file types as ParaView? If not, is there a comprehensive list somewhere saying which types are compatible?

Since Visualizer is using ParaView for the data processing and rendering, everything that ParaView can load, Visualizer can do it as well with only one caveat. Sometime for specific file extension, ParaView will ask you which reader you want to use to read a given file. Visualizer on the other hand will pick one by default without asking the user. If a specific one should be used, that is not the default, you can override it by providing a custom configuration like here using the following argument.

To preload a file/state, you can use that argument

Then you will use the launcher to control those arguments.

That's perfect, thanks for pointing me towards that. I've now been able to preload the datasets and in the prepared state on my local machine.

What I'd like to do is this:

Have a landing page (e.g. website.com/index.html) which lists a few example datasets.
The user then clicks on a link which opens a visualizer page with the correct data. (e.g. website.com/VisualizerData1.html or website.com/Visualizer?data=filename).

When I do it locally I could use the webpage to trigger a script which launches visualizer with the correct --load-file flag. What I can't work out is how to do this using ami-34f3f65e. Or is this not possible, because the ami pre-launches visualizer and therefore cannot pre-load different datasets for parallel sessions? I've tried looking within '/data/...' for where I might add the flag, I thought maybe 'launcher-TEMPLATE.json' but I can't spot the right place.

Thanks in advance for your advice.

It is definitely possible on the AMI as well. It is just that I'm not sure (don't remember) where the launcher config is stored as well as which version of PV+Visualizer that AMI has. You might be on the right track regarding the launcher-TEMPLATE.json.

But I want to make sure you solved all the pieces locally the way I'm thinking with the same version of Visualizer and ParaView.

you edited your launcher-config.json to use some argument to load the data at startup.
you "may need" to edit the index.html to process the URL arguments and add them to the configuration that get sent to the launcher. (It was automatic for some time, then that feature went away and I think it is back, but you may need to enable it manually)

Locally I've only run the basic one session setup.
My hacky 'solution' is to have a shell script which I call and pass a variable to point at the .pvsm file

#!/bin/bash
./bin/pvpython   \
    ./share/paraview-5.4/web/visualizer/server/pvw-visualizer.py  \
    --content ./share/paraview-5.4/web/visualizer/www/               \
    --data $PWD/share/paraview-5.4/data                                      \
    --load-file $PATH_TO_DATA                                                      \
    --port 8080

I don't mind having a different index.html (index1.html, index2.html etc.) for each dataset if that's easier, because there'll only be a limited number of examples made available.

Can you check on that ami whether there is a pvw-launcher.conf in /etc/init/? If so, it would refer to the location of the launcher config, which might be /etc/opt/paraview/launcher.json.

You don't need a different html for each of your data file that you want to load.

Let's solve it on your local system first then we'll deal with AMI later.

Start using a launcher to trigger a new visualizer process
Extend the connection's configuration in the index.html of Visualizer so the launcher can use those extra properties. (We will use that to specify the file to load later)
Customize the command line that the launcher use to start visualizer to leverage that new property that get passed to it.
Leverage the url parameter to fill that configuration with the proper file name.

See this guide as it is exactly doing that but on Windows.

Otherwise you can learn more on the launcher here too.

@scottwittenburg

@scottwittenburg
I can't see a pvw-launcher.conf file, but there is a /etc/init.d/pvw-start.
There's nothing at all in /etc/opt/. On the ami, ParaView is under /data/...

@jourdain
I'll look into using the launcher now.

In the script here do I need to replace paraview-root and apache-root with the correct path, or are these variables?

That file /etc/init.d/pvw-start should just be a text file which should point you to the existing launcher configuration.

@scottwittenburg

@scottwittenburg
Contents of pvw-start

In your use case (local), you don't need to worry about apache. But those need to be replaced by you with something valid.

Moreover, since you are looking at the windows doc, make sure you fix the sessionURL with something like: "sessionURL" : "ws://localhost:${port}/ws",
That way you skip the Apache front-end.

Ok here's my launcher.config, does that look ok?

And modified index.html

Couple of comment

You can not use ~, you need to use full path

"proxy_file" : "/tmp/proxy.txt",

(index.html) Then remove line 13 otherwise you will connect twice...

Ok I've updated to use a full path (and in index.html). By line 13 I assume you meant 13-16, i.e.

  "resources" : [ {
  "host" : "localhost",
  "port_range" : [9001, 9999]
  } ],

What should happen at this point? Do I need to do anything to launch before going to localhost via a browser? If I go to http://localhost:9000/?data=can.ex2 nothing happens. From launcher.config I assume that's the port I should go to.

Ok, so I tried adding "--load-file", "${dataFile}" to line 33 of launcher.json. What else would I need to do?

That's the trick. Did you change it in the config that actually run (the template one + restart?)

No, I hadn't. I did it in the wrong one! Just tried again in template and hard coding can.ex2 into index.html worked. Thanks.

Now to try adding getUrlParameter.

Perfect, it worked. Hopefully by the end of the week I'll have added a few small demo datasets. Then If you're interested I can send you the link to have a look.

Thanks again for the help.

I'm thinking about uploading my datasets to a separate (always up) server, then mounting this on the EC2 instance running visualizer.

My assumption is that this is a bad idea because ParaView expects datasets to potentially be very large and therefore doesn't do unscalable things like make temporary copies or read the whole thing into RAM. If my assumption is correct I'll be depending on how fast the network can download each timestep file, which probably means it would be better to store all data on my EC2 instance to avoid a major hit to performance?

I just wanted to ask for your opinion because I wasn't sure how ParaView handles opened data.

You should put your dataset on an EBS which can then be only mounted on a system at a time.
But that way, you will have better load time as ParaView expect to have those file locally available.
But you will only have one server available with the data at a given time.

If you want both, you can put your data on a server that is always up. But then you have a cron job on the ec2-gpu machine that rsynch them back to the SSD drive of that machine periodically. That way those become local to the ParaView and the read time will always be good.

Using rsync on boot of the viz server is a great idea, thanks.

I've uploaded some example datasets. So if you happen to be online and are interested feel free to have a look here.

Please excuse how basic it is, most of the text etc. are just placeholders for the moment while I build/test functionality. Let me know if you have any thoughts (it'll be up for the next couple of hours).

Thanks for sharing. That looks great... Depending on how big is the data and what you want to do with it, vtk.js could also provides another rendering alternative...

https://kitware.github.io/vtk-js/

Yes, my data will likely be pretty large. Is vtk.js better adapted for larger data? Would I need to convert data to the vtk format?

Other than having a few demo datasets as an example of research I think the main way I was planning to use visualizer was to share research data with collaborators in such a way that they don't need to install software (and may not have appropriate hardware).

If the data is large, vtk.js won't be a good fit as it will require the client (browser) to download the dataset so it can be rendered locally.

In that case, ParaViewWeb is the perfect fit. Once we'll create a new AMI with the latest code, it could be nice/good to update knowing that everything that you did will still apply as the logic didn't change only the UI will be more appealing... ;-)

https://www.paraview.org/wp-content/uploads/2014/04/visualizer.png

Ah ok, useful to know as some datasets are GBs.

ParaViewWeb is really handy. I've even been testing it on my phone. I can envisage being able to use it on a tablet in a poster session for a conference.

I'll definitely be keeping an eye out for the updated ami and will happily test that everything I've done still works. At some point I should learn how to set up apache and things, but I'm very appreciative for the current ami which has all of than ready out of the box.

One little bug I have noticed: For some of the data I uploaded there were timesteps, but it only shows the one that was loaded when I saved the pvsm locally before uploading (i.e. you can't access other timesteps). So I also tried opening the data via ParaViewWeb and all time steps were then accessible, but when I saved the state from PVW it would always open on the first time step. It's minor issue and it may be a version compatibility issue.

Glad to ear that it offer a good solution for your needs.

I would be curious if the latest version of Visualizer and Paraview have similar issue regarding missing timestep when loading the data from a state file instead of the actual dataset files.

Feel free to add a bug so we can keep track of it and fix it...

It's great because I can visualize both CT and FEA data using the same program.

Yes, I think the issue is present on v5.4 because I tried on my local machine before uploading to AWS and that's where I first noticed it. I'll try looking in a bit more detail to make sure, if I'm correct I'll start a new issue.

Navigation Menu

Repository navigation

Auto-load data and 'scene' (or state) on launch #47

18 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Auto-load data and 'scene' (or state) on launch #47

Description

Activity

jourdain commented on Jan 11, 2018

llionevans commented on Jan 12, 2018

jourdain commented on Jan 12, 2018

llionevans commented on Jan 12, 2018

scottwittenburg commented on Jan 12, 2018

jourdain commented on Jan 12, 2018

llionevans commented on Jan 12, 2018

llionevans commented on Jan 12, 2018

scottwittenburg commented on Jan 12, 2018

llionevans commented on Jan 12, 2018

jourdain commented on Jan 12, 2018

llionevans commented on Jan 12, 2018

llionevans commented on Jan 12, 2018

jourdain commented on Jan 12, 2018

llionevans commented on Jan 12, 2018

18 remaining items

llionevans commented on Jan 16, 2018

jourdain commented on Jan 16, 2018

llionevans commented on Jan 16, 2018

llionevans commented on Jan 16, 2018

llionevans commented on Jan 18, 2018

jourdain commented on Jan 18, 2018

llionevans commented on Jan 18, 2018

llionevans commented on Jan 22, 2018

jourdain commented on Jan 22, 2018

jourdain commented on Jan 22, 2018

llionevans commented on Jan 22, 2018

jourdain commented on Jan 22, 2018

llionevans commented on Jan 22, 2018

jourdain commented on Jan 22, 2018

llionevans commented on Jan 22, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions