Lesson 3, Part A: Show the data as it is acquired#
In this lesson, we’ll show how to use the tools provided with Bluesky to show the data as it is acquired using both a table representation and a graphical view, as well. These capabilities are provided by using callbacks. In lessons 1 and 2, we wrote our own simple callback to view the documents that come from the RunEngine during execution of a plan. Quickly, the data became too complex for simple viewing.
The LiveTable and LivePlot callbacks provide a table and graphical view of the data from the plan. We’ll get to those first. Later, we’ll show the BestEffortCallback, which combines both those callbacks plus a little more. For routine work, we’ll want to use BestEffortCallback all the time. We’ll show how to make that happen so we set it and forget about it.
note: This tutorial expects to find an EPICS IOC on the local network configured as a synApps xxx IOC with prefix sky:. A docker container is available to provide this IOC. See this URL for instructions: prjemian/epics-docker
Starting with the configuration from lessons 1 and 2, we first group the imports together as is common Python practice:
[1]:
from ophyd import EpicsMotor
from ophyd.scaler import ScalerCH
from bluesky import RunEngine
import bluesky.plans as bp
from apstools.devices import use_EPICS_scaler_channels
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[1], line 1
----> 1 from ophyd import EpicsMotor
2 from ophyd.scaler import ScalerCH
3 from bluesky import RunEngine
ModuleNotFoundError: No module named 'ophyd'
Next, make a RunEngine (for scanning) and connect our motor and scaler
[2]:
RE = RunEngine({})
P = "sky:"
m1 = EpicsMotor(f"{P}m1", name="m1")
scaler = ScalerCH(f"{P}scaler1", name="scaler")
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[2], line 1
----> 1 RE = RunEngine({})
3 P = "sky:"
4 m1 = EpicsMotor(f"{P}m1", name="m1")
NameError: name 'RunEngine' is not defined
Reconfigure the scaler for channel names, set the counting time to 0.5 s, and read the scaler values.
[3]:
scaler.channels.chan01.chname.put("clock")
scaler.channels.chan02.chname.put("I0")
scaler.channels.chan03.chname.put("scint")
scaler.preset_time.put(0.4)
scaler.select_channels(None)
scaler.read()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[3], line 1
----> 1 scaler.channels.chan01.chname.put("clock")
2 scaler.channels.chan02.chname.put("I0")
3 scaler.channels.chan03.chname.put("scint")
NameError: name 'scaler' is not defined
Showing the data#
In lessons 1 and 2, we wrote a callback routine that printed information as the scan prpogressed (that is, we printed select content from the stream of documents emitted by the RunEngine while executing a plan). But our callback was simple and we found there is a lot of content in the documents from the RunEngine.
The simplest example of a Bluesky callback is the print function. We want a callback function that understands our data and uses reasonable assumptions to show that data as it is being acquired.
One method to display our data is in a table that updates as the scan progresses. We’ll import the LiveTable callback from the Bluesky library:
[4]:
from bluesky.callbacks import LiveTable
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[4], line 1
----> 1 from bluesky.callbacks import LiveTable
ModuleNotFoundError: No module named 'bluesky'
LiveTable() shows acquired data as a plan is executed. The argument is the list of detectors to show in the table. First, we’ll count the scaler 5 times.
[5]:
RE(bp.count([scaler], 5), LiveTable([scaler]))
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[5], line 1
----> 1 RE(bp.count([scaler], 5), LiveTable([scaler]))
NameError: name 'RE' is not defined
You see columns for the data collection sequence number, the time of collection, and each of the named scaler channels. At the end, the short form for the scan’s uid is shown as well as scan num which is a more convenient reference to the scan. The user has control to set or reset scan num so do not rely on that number to be unique.
Next, we’ll scan with motor and scaler, as we did in lesson 2, displaying the acquired data in a LiveTable.
[6]:
RE(bp.scan([scaler], m1, 1, 5, 5), LiveTable([m1, scaler]))
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[6], line 1
----> 1 RE(bp.scan([scaler], m1, 1, 5, 5), LiveTable([m1, scaler]))
NameError: name 'RE' is not defined
In addition to the data columns from count above, the motor position (both where the motor reported as its position and where the motor was told to go, respectively) are shown.
There is a callback routine that will plot the data as it is acquired. When starting graphics, it is necessary to first initialize the graphics manager of the display. The setup is specific to the graphics manager. For command line or python program use, see https://blueskyproject.io/bluesky/callbacks.html#aside-making-plots-update-live.
For jupyter notebooks:
[7]:
%matplotlib notebook
from bluesky.utils import install_nb_kicker
install_nb_kicker()
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[7], line 1
----> 1 get_ipython().run_line_magic('matplotlib', 'notebook')
2 from bluesky.utils import install_nb_kicker
3 install_nb_kicker()
File /opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/IPython/core/interactiveshell.py:2480, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth)
2478 kwargs['local_ns'] = self.get_local_scope(stack_depth)
2479 with self.builtin_trap:
-> 2480 result = fn(*args, **kwargs)
2482 # The code below prevents the output from being displayed
2483 # when using magics with decorator @output_can_be_silenced
2484 # when the last Python token in the expression is a ';'.
2485 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):
File /opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/IPython/core/magics/pylab.py:103, in PylabMagics.matplotlib(self, line)
98 print(
99 "Available matplotlib backends: %s"
100 % _list_matplotlib_backends_and_gui_loops()
101 )
102 else:
--> 103 gui, backend = self.shell.enable_matplotlib(args.gui)
104 self._show_matplotlib_backend(args.gui, backend)
File /opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/IPython/core/interactiveshell.py:3665, in InteractiveShell.enable_matplotlib(self, gui)
3662 import matplotlib_inline.backend_inline
3664 from IPython.core import pylabtools as pt
-> 3665 gui, backend = pt.find_gui_and_backend(gui, self.pylab_gui_select)
3667 if gui != None:
3668 # If we have our first gui selection, store it
3669 if self.pylab_gui_select is None:
File /opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/IPython/core/pylabtools.py:338, in find_gui_and_backend(gui, gui_select)
321 def find_gui_and_backend(gui=None, gui_select=None):
322 """Given a gui string return the gui and mpl backend.
323
324 Parameters
(...)
335 'WXAgg','Qt4Agg','module://matplotlib_inline.backend_inline','agg').
336 """
--> 338 import matplotlib
340 if _matplotlib_manages_backends():
341 backend_registry = matplotlib.backends.registry.backend_registry
ModuleNotFoundError: No module named 'matplotlib'
We’ll import the LivePlot callback from the Bluesky library:
[8]:
from bluesky.callbacks import LivePlot
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[8], line 1
----> 1 from bluesky.callbacks import LivePlot
ModuleNotFoundError: No module named 'bluesky'
Count the scaler 5 times. We’ll just plot the scint signal.
[9]:
RE(bp.count([scaler],num=5), LivePlot("scint"))
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[9], line 1
----> 1 RE(bp.count([scaler],num=5), LivePlot("scint"))
NameError: name 'RE' is not defined
To scan, we need to tell LivePlot to plot `scint’ vs. the motor:
[10]:
RE(bp.scan([scaler], m1, 1, 5, 5), LivePlot("scint", "m1"))
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[10], line 1
----> 1 RE(bp.scan([scaler], m1, 1, 5, 5), LivePlot("scint", "m1"))
NameError: name 'RE' is not defined
Both the table and the plot are very useful diagnostics for routine use. They have been combined in the Best-Efforts Callback which provides best-effort plots and visualization for any plan. It uses user-configurable information that is part of every ophyd device to make reasonable assumptions about what information is appropriate to display in the context of the current plan.
We’ll import the BestEffortCallback callback from the Bluesky library:
[11]:
from bluesky.callbacks.best_effort import BestEffortCallback
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[11], line 1
----> 1 from bluesky.callbacks.best_effort import BestEffortCallback
ModuleNotFoundError: No module named 'bluesky'
Count the scaler 5 times:
[12]:
RE(bp.count([scaler], num=5), BestEffortCallback())
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[12], line 1
----> 1 RE(bp.count([scaler], num=5), BestEffortCallback())
NameError: name 'RE' is not defined
You see both the LiveTable and the LivePlot output tangled up here in the jupyter notebook. Each is created on demand and then updated as the plan progresses. When executing in a command line environment, the LivePlot is shown in a separate window.
Repeat the same scan, noting that we do not need to inform the callback what to display:
[13]:
RE(bp.scan([scaler], m1, 1, 5, 5), BestEffortCallback())
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[13], line 1
----> 1 RE(bp.scan([scaler], m1, 1, 5, 5), BestEffortCallback())
NameError: name 'RE' is not defined
Because this is such a useful tool, we want to make this callback happen all the time. The RunEngine manages a list of such callbacks. We subscribe the BestEffortCallback:
[14]:
RE.subscribe(BestEffortCallback())
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[14], line 1
----> 1 RE.subscribe(BestEffortCallback())
NameError: name 'RE' is not defined
Repeat the count of the scaler (without adding the callback in the command):
In jupyter notebook, we can see the LiveTable after our scan command. To see the LivePlot, we have to look up a few cells, where the plots of scaler channels vs. m1 are shown, our latest data identified by scan num in the legend.RE(bp.count([scaler], num=5))
[15]:
RE(bp.count([scaler], num=5))
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[15], line 1
----> 1 RE(bp.count([scaler], num=5))
NameError: name 'RE' is not defined
In jupyter notebook, we can see the LiveTable after our count command. To see the LivePlot, we have to look up a few cells, where the plots of scaler channels vs. time are shown, our latest data identified by scan num in the legend.
Then, repeat the scan (again, without adding the callback in the command):
[16]:
RE(bp.scan([scaler], m1, 1, 5, 5))
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[16], line 1
----> 1 RE(bp.scan([scaler], m1, 1, 5, 5))
NameError: name 'RE' is not defined
In jupyter notebook, we can see the LiveTable after our scan command. Notice that the number of columns displayed is less than when we called LiveTable ourselves. To see the LivePlot, we have to look up a few cells, where the plots of scaler channels vs. m1 are shown, our latest data identified by scan num in the legend.
Summary#
We’ll show this code as a python program:
#!/usr/bin/env python
"lesson 3: Show the data as it is acquired"
from ophyd import EpicsMotor
from ophyd.scaler import ScalerCH
from bluesky import RunEngine
import bluesky.plans as bp
from bluesky.callbacks import LiveTable
from bluesky.callbacks import LivePlot
from bluesky.callbacks.best_effort import BestEffortCallback
from apstools.devices import use_EPICS_scaler_channels
%matplotlib notebook
from bluesky.utils import install_qt_kicker
install_qt_kicker()
RE = RunEngine({})
P = "sky:"
m1 = EpicsMotor(f"{P}m1", name="m1")
scaler = ScalerCH(f"{P}scaler1", name="scaler")
m1.wait_for_connection()
scaler.wait_for_connection()
scaler.preset_time.put(0.4)
scaler.select_channels(None)
print(scaler.read())
RE(bp.count([scaler], num=5), LiveTable([scaler]))
RE(bp.scan([scaler], m1, 1, 5, 5), LiveTable([m1, scaler]))
RE(bp.count([scaler], num=5), LivePlot("scint"))
RE(bp.scan([scaler], m1, 1, 5, 5), LivePlot("scint", "m1"))
RE(bp.count([scaler], num=5), BestEffortCallback())
RE(bp.scan([scaler], m1, 1, 5, 5), BestEffortCallback())
RE.subscribe(BestEffortCallback())
RE(bp.count([scaler], num=5))
RE(bp.scan([scaler], m1, 1, 5, 5))