Images, Darks, & whites with EPICS area detector, ophyd, and Bluesky#
In some scientific data acquisition processes, an area detector acquires a sequence of image frames and stores them as a sequence in a data file. In tomography and other processes, the frames are associated with certain imaging conditions, such as when the illumination is on and the sample is in view (known as an image or data frame) or the sample is not in view (a flat, flood, or white frame) or when the illumination is off (a dark or background frame).
This document describes how to configure the Bluesky software to write such HDF5 files when using EPICS and the Area Detector support.
The HDF5 File Writer plugin can be configured to save image frames into separate datasets within the same HDF5 data file. The selection of frame type (image frame, background/dark frame, white/flat frame) is made by use of an existing PV in area detector: $(P):cam1:FrameType which is an mbbo record. In ophyd, the readback version of this PV:
$(P):cam1:FrameType_RBV is used to define operational values for the ophyd device. Be sure to configure the readback PV with the same values.
meaning |
PV value |
PV field |
default |
||
|---|---|---|---|---|---|
image frame |
0 |
|
|
|
|
background/dark frame |
1 |
|
|
|
|
white/flat frame |
2 |
|
|
|
|
not used |
3 |
|
|
|
|
not used |
4 |
|
|
|
|
The values will be used as the HDF5 addresses to store that type of frame. With the chosen format (dxchange or NeXus), set the fields of both these PVs ($(P):cam1:FrameType and $(P):cam1:FrameType_RBV).
In NeXus, detector data is stored within the instrument group, traditionally at /entry/instrument/detector/. We then hard link the field data, dark, and white fields to the /entry/data/ group. The hard links provide a shorter HDF5 address. We use this shorter address in EPICS, to fit within the maximum 25 characters allowed by the EPICS mbbo record. We write the data into
the /entry/data group and hard link it (in the layout.xml file) to the instrument group.
Tip: Whether you use dxchange or NeXus, be sure the fields of the cam1:FrameType PV are put in the IOC’s autosave configuration so they are restored when the IOC is restarted!
The area detector attributes XML file needs the cam1:FrameType selection PV included in its list. We’ll call it HDF5FrameLocation so we can use the same name in the layout file:
<Attribute
name="HDF5FrameLocation"
type="EPICS_PV"
source="13SIM1:cam1:FrameType"
dbrtype="DBR_STRING"
description="HDF5 address for this frame"/>
The HDF5 layout XML file will refer to this HDF5FrameLocation attribute in the setup (add this to the XML file just after the opening hdf5_layout and before the first group element).
<global name="detector_data_destination" ndattribute="HDF5FrameLocation" />
The name detector_data_destination is hard-coded in the source code of the HDF5 file writer.
In the Bluesky plan, write the frame type before acquisition with a value from this table:
use this value |
which means a frame of this type |
|---|---|
|
image |
|
dark |
|
white |
In any following acquisition(s), the HDF5 file writer will direct the image frame to the dataset as specified by the ZRST, ONST, or TWST field, respectively.
Setup#
Try acquiring three different frame types into the same HDF5 file with the ADSimDetector IOC. Use the NeXus format. The procedures are similar for the data exchange format.
Notebook-specific details#
These details are specific to the area detector running for this notebook.
variable |
comments |
|---|---|
|
IOC prefix for the ADSimdetector IOC |
|
root directory of this IOC as mounted on the workstation for this notebook |
[1]:
import pathlib
ad_prefix = "ad:"
IOC_MOUNT_POINT = pathlib.Path("/tmp/docker_ioc/iocad")
XML Attributes File#
Here, we create the custom /tmp/attributes.xml (starting from the XML layout file supplied with the ADSimDetector). Rather than use an XML library to write this file, we’ll create all the content here as text and write the file with the usual text file tools. Then, we’ll check that the file exists.
Note: This is the part of the file that connects the cam1:FrameType PV with an attribute to be used in the layout file. We can choose any unique name for this attribute (within the names allowed by XML) but we must use the same name in both the attributes and layout XML files. Here, we choose the name HDF5FrameLocation .
<Attribute
name="HDF5FrameLocation"
type="EPICS_PV"
source="{ad_prefix}cam1:FrameType"
dbrtype="DBR_STRING"
description="HDF5 address for this frame"
/>
In a later step, we’ll tell the HDF5 plugin to use it.
NOTE: Very important that you set the attributes file in the cam, and not the hdf1 plugin!
[2]:
xml_file = IOC_MOUNT_POINT / "tmp" / "attributes.xml" # IOC sees: /tmp/attributes.xml
XML_ATTRIBUTES = f"""
<?xml version="1.0" standalone="no" ?>
<Attributes
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="../../../../../ADCore/XML_schema/NDAttributes.xsd"
>
<Attribute name="AcquireTime" type="EPICS_PV" source="{ad_prefix}cam1:AcquireTime" dbrtype="DBR_NATIVE" description="Camera acquire time"/>
<Attribute name="HDF5FrameLocation" type="EPICS_PV" source="{ad_prefix}cam1:FrameType" dbrtype="DBR_STRING" description="HDF5 address for this frame"/>
<Attribute name="ImageCounter" type="PARAM" source="ARRAY_COUNTER" datatype="INT" description="Image counter"/>
<Attribute name="AttributesFileParam" type="PARAM" source="ND_ATTRIBUTES_FILE" datatype="STRING" description="Attributes file param"/>
<Attribute name="CameraModel" type="PARAM" source="MODEL" datatype="STRING" description="Camera model"/>
<Attribute name="CameraManufacturer" type="PARAM" source="MANUFACTURER" datatype="STRING" description="Camera manufacturer"/>
</Attributes>
"""
with open(xml_file, "w") as f:
f.write(XML_ATTRIBUTES.strip())
print(f"{xml_file.exists()=}\n{xml_file=}")
xml_file.exists()=True
xml_file=PosixPath('/tmp/docker_ioc/iocad/tmp/attributes.xml')
XML Layout File#
Here, we create the custom /tmp/layout.xml (starting from the XML layout file supplied with the ADSimDetector). Rather than use an XML library to write this file, we’ll create all the content here as text and write the file with the usual text file tools. Then, we’ll check that the file exists.
Note: This is the part of the file that directs each frame type to its configured field:
<global name="detector_data_destination" ndattribute="HDF5FrameLocation" />
The detector_data_destination name is compiled into the HDF5 writer plugin and is required for proper operation. The HDF5FrameLocation coordinates with the same name in the attributes above.
In a later step, we’ll tell the HDF5 plugin to use it.
[3]:
xml_file = IOC_MOUNT_POINT / "tmp" / "layout.xml" # IOC sees: /tmp/layout.xml
XML_LAYOUT2 = """
<?xml version="1.0" standalone="no" ?>
<hdf5_layout
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="../../../../../ADCore/XML_schema/hdf5_xml_layout_schema.xsd"
>
<global name="detector_data_destination" ndattribute="HDF5FrameLocation" />
<group name="entry">
<attribute name="NX_class" source="constant" value="NXentry" type="string" />
<attribute name="default" source="constant" value="data" type="string" />
<group name="data">
<attribute name="NX_class" source="constant" value="NXdata" type="string" />
<attribute name="signal" source="constant" value="data" type="string" />
<dataset name="data" source="detector">
<attribute name="units" source="constant" value="counts" type="string" />
<attribute name="description" source="constant" value="image frame(s)" type="string" />
<attribute name="target" source="constant" value="/entry/data/data" type="string" />
</dataset>
<dataset name="dark" source="detector">
<attribute name="units" source="constant" value="counts" type="string" />
<attribute name="description" source="constant" value="dark (background) frame(s)" type="string" />
<attribute name="target" source="constant" value="/entry/data/dark" type="string" />
</dataset>
<dataset name="white" source="detector">
<attribute name="units" source="constant" value="counts" type="string" />
<attribute name="description" source="constant" value="white (flat) frame(s)" type="string" />
<attribute name="target" source="constant" value="/entry/data/white" type="string" />
</dataset>
</group> <!-- end group data -->
<group name="instrument">
<attribute name="NX_class" source="constant" value="NXinstrument" type="string" />
<group name="detector">
<attribute name="NX_class" source="constant" value="NXdetector" type="string" />
<hardlink name="data" target="/entry/data/data" />
<hardlink name="dark" target="/entry/data/dark" />
<hardlink name="white" target="/entry/data/white" />
</group> <!-- end group detector -->
<group name="NDAttributes" ndattr_default="true">
<attribute name="NX_class" source="constant" value="NXcollection" type="string" />
</group> <!-- end group NDAttribute (default) -->
</group> <!-- end group instrument -->
</group> <!-- end group entry -->
</hdf5_layout>
"""
with open(xml_file, "w") as f:
f.write(XML_LAYOUT2.strip())
print(f"{xml_file.exists()=}\n{xml_file=}")
xml_file.exists()=True
xml_file=PosixPath('/tmp/docker_ioc/iocad/tmp/layout.xml')
Ophyd Device Setup#
[4]:
from apstools.devices import CamMixin_V34
from apstools.devices import SingleTrigger_V34
from ophyd import Component
from ophyd import Device
from ophyd import EpicsSignal
from ophyd import EpicsSignalRO
from ophyd import EpicsSignalWithRBV
from ophyd import SimDetector
from ophyd.areadetector import SimDetectorCam
from ophyd.areadetector.plugins import HDF5Plugin_V34
from ophyd.areadetector.plugins import ImagePlugin_V34
from ophyd.areadetector.plugins import PvaPlugin_V34
class MySimDetectorCam(CamMixin_V34, SimDetectorCam):
nd_attr_status = Component(EpicsSignalRO, "NDAttributesStatus", kind="omitted", string=True)
class MyHDF5Plugin(HDF5Plugin_V34):
layout_filename = Component(EpicsSignal, "XMLFileName", kind="config", string=True)
layout_filename_valid = Component(EpicsSignal, "XMLValid_RBV", kind="omitted", string=True)
nd_attr_status = Component(EpicsSignal, "NDAttributesStatus", kind="omitted", string=True)
class MyDetector(SingleTrigger_V34, SimDetector):
cam = Component(MySimDetectorCam, suffix="cam1:")
hdf1 = Component(MyHDF5Plugin, suffix="HDF1:")
image = Component(ImagePlugin_V34, suffix="image1:")
pva1 = Component(PvaPlugin_V34, suffix="Pva1:")
Support Functions#
[5]:
import h5py
IMAGE = 0
DARK = 1
WHITE = 2
def set_ad_count_time(det, exposure=1, period=1):
det.cam.stage_sigs["acquire_time"] = exposure
det.cam.stage_sigs["acquire_period"] = period
def ad_setup(det, nframes=1):
"""Make this a function. We'll use again later."""
det.cam.acquire.put(0)
det.cam.frame_type.kind = "config"
det.hdf1.capture.put(0)
det.hdf1.compression.put("zlib") # better than `"None"` (default)
det.hdf1.create_directory.put(-5)
det.hdf1.file_name.put("test_image")
det.hdf1.file_path.put("/tmp")
det.hdf1.kind = 3 # config | normal
if "compression" in det.hdf1.stage_sigs:
det.hdf1.stage_sigs.pop("create_directory")
# The plugins do not block, the cam must wait for the plugins to finish.
for nm in det.component_names:
obj = getattr(det, nm)
if "blocking_callbacks" in dir(obj): # is it a plugin?
obj.stage_sigs["blocking_callbacks"] = "No"
det.cam.stage_sigs["wait_for_plugins"] = "Yes"
det.cam.stage_sigs["num_images"] = nframes
det.hdf1.stage_sigs["num_capture"] = 0 # capture ALL frames received
det.hdf1.stage_sigs["auto_increment"] = "Yes"
det.hdf1.stage_sigs["auto_save"] = "Yes"
det.hdf1.stage_sigs["file_template"] = "%s%s_%4.4d.h5"
det.hdf1.stage_sigs["file_write_mode"] = "Stream"
det.hdf1.stage_sigs["store_attr"] = "Yes" # need in this notebook
det.hdf1.stage_sigs["store_perform"] = "No" # optional
det.hdf1.stage_sigs["auto_increment"] = "Yes"
det.hdf1.stage_sigs["capture"] = 1 # ALWAYS last
det.hdf1.stage_sigs.move_to_end("capture") # ... just in case
# Clear these settings __for this demo__.
det.hdf1.layout_filename.put("")
det.cam.nd_attributes_file.put("")
def check_adplugin_primed(plugin, allow_priming=True):
from apstools.devices import AD_plugin_primed
from apstools.devices import AD_prime_plugin2
# this step is needed for ophyd
if not AD_plugin_primed(plugin):
if allow_priming:
print(f"Priming {plugin.dotted_name}")
AD_prime_plugin2(plugin)
else:
raise RuntimeError(
f"Detector plugin '{plugin.dotted_name}' must be primed first."
)
def dict_to_table(d, printing=True):
import pyRestTable
if len(d) == 0:
return
table = pyRestTable.Table()
table.labels = "key value".split()
table.rows = [[k, v] for k, v in d.items()]
if printing:
print(table)
else:
return table
def readings_to_table(d, printing=True):
import pyRestTable
if len(d) == 0:
return
labels = sorted(set([k for v in d.values() for k in v.keys()]))
table = pyRestTable.Table()
# fmt: off
table.labels = [ "name", ] + list(labels)
for k, reading in d.items():
row = [k, ] + [reading.get(r, "") for r in labels]
table.addRow(row)
# fmt: on
if printing:
print(table)
else:
return table
def print_overview(device):
cfg = device.describe_configuration()
for k, readings in device.read_configuration().items():
if k not in cfg:
cfg[k] = readings
else:
cfg[k].update(readings)
if len(cfg) > 0:
print(f"'{device.name}' configuration:")
readings_to_table(cfg)
def practice_device_staging(device):
print(f"Before staging '{device.name}'")
device.stage()
print(f"Device '{device.name}' staged")
device.unstage()
print(f"Device '{device.name}' unstaged")
def local_h5_file(det):
ioc_file = det.hdf1.full_file_name.get()
return IOC_MOUNT_POINT / ioc_file.lstrip("/")
def check_h5_file(det):
hfile = local_h5_file(det)
print(f"{hfile.exists()=} {hfile.name=}")
addr = "/entry/instrument/NDAttributes"
with h5py.File(hfile, "r") as root:
group = root[addr]
print(f"{len(group)=} {group=}")
print(f"members: {[k for k in group]}")
def h5_overview(det):
h5_file = local_h5_file(det)
print(f"{h5_file=}")
with h5py.File(h5_file, "r") as NeXus_data:
print(f"{('/entry/data' in NeXus_data)=}")
print(f"{('/entry/data/dark' in NeXus_data)=}")
print(f"{('/entry/data/data' in NeXus_data)=}")
print(f"{('/entry/data/white' in NeXus_data)=}")
print(f"{('/entry/instrument' in NeXus_data)=}")
print(f"{('/entry/instrument/detector' in NeXus_data)=}")
print(f"{('/entry/instrument/detector/dark' in NeXus_data)=}")
print(f"{('/entry/instrument/detector/data' in NeXus_data)=}")
print(f"{('/entry/instrument/detector/white' in NeXus_data)=}")
print(f"{('/entry/instrument/NDAttributes' in NeXus_data)=}")
print(f"{('/entry/instrument/NDAttributes/HDF5FrameLocation' in NeXus_data)=}")
for addr in "dark data white".split():
try:
ds = NeXus_data[f"/entry/data/{addr}"]
print(f"{ds=} {ds.shape=}")
except KeyError:
pass
frame_type = det.cam.frame_type.get()
print(
f"frame_type: PV='{det.cam.frame_type.pvname}'"
f"\n value={frame_type}"
f" ({det.cam.frame_type.enum_strs[frame_type]})"
f"\n choices={det.cam.frame_type.enum_strs}"
)
print(f"{h5_file=}")
def setup_frame_type():
class FrameType(Device):
zero = Component(EpicsSignal, ".ZRST", string=True)
one = Component(EpicsSignal, ".ONST", string=True)
two = Component(EpicsSignal, ".TWST", string=True)
three = Component(EpicsSignal, ".THST", string=True)
for pv in ("FrameType", ):
o = FrameType(f"{ad_prefix}cam1:{pv}", name="o")
o.wait_for_connection()
o.zero.put("/entry/data/data")
o.one.put("/entry/data/dark")
o.two.put("/entry/data/white")
o.three.put("")
Configure PV for the Frame Type#
Configure our PV(s) for the NeXus addresses we want to use. We must reconnect our ophyd object after our change to the EPICS PVs to pick up this change. (The choices that the Python objects sees are only updated when the cam1:FrameType PV is first connected.)
Must call setup_frame_type() before creating detector object, so the Python detector object picks up on the different string values. The EPICS IOC gets the correct string directly from the cam1:FrameType PV.
[6]:
setup_frame_type()
Create Detector Object#
… and connect with the Area Detector IOC. Set the counting time per frame (something short). Prime (push an image frame from the camera to the plugin) the HDF5 plugin, if found necessary.
[7]:
adsimdet = MyDetector(ad_prefix, name="det")
adsimdet.wait_for_connection()
NUM_FRAMES = 5
set_ad_count_time(adsimdet, exposure=0.02, period=0.1)
ad_setup(adsimdet, nframes=NUM_FRAMES)
check_adplugin_primed(adsimdet.hdf1)
Priming hdf1
Create RunEngine#
[8]:
import apstools
import bluesky
import databroker
from bluesky import plans as bp
cat = databroker.temp().v2
RE = bluesky.RunEngine()
RE.subscribe(cat.v1.insert)
RE.md["title"] = "images, darks, & flats"
RE.md["versions"]["apstools"] = apstools.__version__
RE.md["repository"] = "bluesky_training"
RE.md["notebook"] = "images_darks_flats"
Review the Configurations#
[9]:
print("RunEngine metadata")
dict_to_table(RE.md)
for o in [adsimdet, adsimdet.cam, adsimdet.hdf1]:
nm = f"adsimdet.{o.attr_name}".rstrip(".")
print(f"'{nm}.stage_sigs' stage_sigs")
dict_to_table(o.stage_sigs)
print_overview(adsimdet)
RunEngine metadata
========== =============================================================
key value
========== =============================================================
versions {'ophyd': '1.7.0', 'bluesky': '1.10.0', 'apstools': '1.6.15'}
title images, darks, & flats
repository bluesky_training
notebook images_darks_flats
========== =============================================================
'adsimdet.stage_sigs' stage_sigs
============== =====
key value
============== =====
cam.acquire 0
cam.image_mode 1
============== =====
'adsimdet.cam.stage_sigs' stage_sigs
================ =====
key value
================ =====
acquire_time 0.02
acquire_period 0.1
wait_for_plugins Yes
num_images 5
================ =====
'adsimdet.hdf1.stage_sigs' stage_sigs
========================== =============
key value
========================== =============
enable 1
blocking_callbacks No
parent.cam.array_callbacks 1
num_capture 0
auto_increment Yes
auto_save Yes
file_template %s%s_%4.4d.h5
file_write_mode Stream
store_attr Yes
store_perform No
capture 1
========================== =============
'det' configuration:
====================== ======= ======================================================= ================ ========= ===== ============================ ================= ===== ================ ==================
name dtype enum_strs lower_ctrl_limit precision shape source timestamp units upper_ctrl_limit value
====================== ======= ======================================================= ================ ========= ===== ============================ ================= ===== ================ ==================
det_cam_acquire_period number 0.0 3 [] PV:ad:cam1:AcquirePeriod_RBV 1681423107.715916 0.0 0.005
det_cam_acquire_time number 0.0 3 [] PV:ad:cam1:AcquireTime_RBV 1681423107.818108 0.0 0.001
det_cam_frame_type integer ('Normal', 'Background', 'FlatField', 'DblCorrelation') None [] PV:ad:cam1:FrameType_RBV 1681423053.905152 None None 0
det_cam_image_mode integer ('Single', 'Multiple', 'Continuous') None [] PV:ad:cam1:ImageMode_RBV 1681423108.021729 None None 2
det_cam_manufacturer string None [] PV:ad:cam1:Manufacturer_RBV 1681423053.904149 None None Simulated detector
det_cam_model string None [] PV:ad:cam1:Model_RBV 1681423053.904223 None None Basic simulator
det_cam_num_exposures integer 0 [] PV:ad:cam1:NumExposures_RBV 1681423053.90519 0 1
det_cam_num_images integer 0 [] PV:ad:cam1:NumImages_RBV 1681423053.904953 0 100
det_cam_trigger_mode integer ('Internal', 'External') None [] PV:ad:cam1:TriggerMode_RBV 1681423053.90516 None None 0
====================== ======= ======================================================= ================ ========= ===== ============================ ================= ===== ================ ==================
Acquire Images with Standard bp.count() Plan#
To understand what the custom frame type support will do for us, we first demonstrate image acquisition when the custom support is not yet installed. When a custom layout file is not configured in the HDF5 plugin, the standard layout (NeXus schema) writes all image frames into the /entry/data/data field of the HDF5 file.
[10]:
RE(bp.count([adsimdet]))
[10]:
('adb1a4a5-19ce-4302-b18b-76eb1c7a4289',)
After the acquisition, check the output file. This is not a great test since we are only writing a single image type. Just check that HDF5FrameLocation is found in the list of members and that the expected data group is present. Only the test results for the data frames should be True (tests for the paths to those components should also be True).
[11]:
print(f"{local_h5_file(adsimdet)}")
check_h5_file(adsimdet)
h5_overview(adsimdet)
/tmp/docker_ioc/iocad/tmp/test_image_0000.h5
hfile.exists()=True hfile.name='test_image_0000.h5'
len(group)=4 group=<HDF5 group "/entry/instrument/NDAttributes" (4 members)>
members: ['NDArrayEpicsTSSec', 'NDArrayEpicsTSnSec', 'NDArrayTimeStamp', 'NDArrayUniqueId']
h5_file=PosixPath('/tmp/docker_ioc/iocad/tmp/test_image_0000.h5')
('/entry/data' in NeXus_data)=True
('/entry/data/dark' in NeXus_data)=False
('/entry/data/data' in NeXus_data)=True
('/entry/data/white' in NeXus_data)=False
('/entry/instrument' in NeXus_data)=True
('/entry/instrument/detector' in NeXus_data)=True
('/entry/instrument/detector/dark' in NeXus_data)=False
('/entry/instrument/detector/data' in NeXus_data)=True
('/entry/instrument/detector/white' in NeXus_data)=False
('/entry/instrument/NDAttributes' in NeXus_data)=True
('/entry/instrument/NDAttributes/HDF5FrameLocation' in NeXus_data)=False
ds=<HDF5 dataset "data": shape (5, 1024, 1024), type "|u1"> ds.shape=(5, 1024, 1024)
frame_type: PV='ad:cam1:FrameType_RBV'
value=0 (Normal)
choices=('Normal', 'Background', 'FlatField', 'DblCorrelation')
h5_file=PosixPath('/tmp/docker_ioc/iocad/tmp/test_image_0000.h5')
Use Ophyd to Acquire Image Frames#
Test the acquisition of the different frame types.
Install Support for Custom Frame Types#
Previously we wrote attributes and layout files for the IOC to use. Configure the IOC to use these files now. With these files, the HDF5 writer will use the cam1:FrameType PV, configured above, to direct each frame to its configured HDF5 address in the file.
NOTE: Very important that you set the attributes file in the cam, and not the hdf1 plugin!
[12]:
adsimdet.cam.nd_attributes_file.put("/tmp/attributes.xml")
print(f"{adsimdet.cam.nd_attr_status.get() = }")
adsimdet.hdf1.layout_filename.put("/tmp/layout.xml")
print(f"{adsimdet.hdf1.layout_filename_valid.get()=}")
adsimdet.cam.nd_attr_status.get() = 'Attributes file OK'
adsimdet.hdf1.layout_filename_valid.get()='Yes'
Start Data Acquisition#
In staging, ophyd sets the various detector PVs to the values to be used for the next data acquisition. The configuration is stored in the various stage_sigs dictionaries printed above.
[13]:
adsimdet.stage()
adsimdet.cam.image_mode.put("Multiple")
Collect Different Frame Types#
Since execution by the RunEngine may appear to be opaque to the user, let’s bypass the RunEngine and collect the different frame types using pure ophyd commands.
For the different image types, first set the frame type, then tell the camera to acquire. The camera has been configured to wait for all plugins to finish before it sets detector state to idle. The responder() function is subscribed to any updates of the detector state PV. A Status() object is used to wait for the PV to reach the value of "Idle" (or 0).
[14]:
from ophyd.status import Status
def acquire(det, frame_type, nframes):
acquisition = Status()
def responder(**kwargs):
"""Called when subscribed signal changes."""
if kwargs.get("value") in (0, "Idle") and not acquisition.done:
acquisition.set_finished()
det.cam.frame_type.put(frame_type)
det.cam.num_images.put(nframes)
det.cam.acquire.put(1)
det.cam.detector_state.subscribe(responder)
acquisition.wait()
det.cam.detector_state.unsubscribe(responder)
We mix up the sequence of images, darks, and whites to demonstrate that the order of these does not matter. Each image type will be directed to the corresponding HDF5 dataset as it is collected.
Collect one image frame
[15]:
acquire(adsimdet, IMAGE, 1)
Collect two white frames
[16]:
acquire(adsimdet, WHITE, 2)
Collect three dark frames
[17]:
acquire(adsimdet, DARK, 3)
Collect five more image frames
[18]:
acquire(adsimdet, IMAGE, 5)
End Data Acquisition#
Unstaging reverses the stage() step above, restoring PVs to their prior values.
[19]:
adsimdet.unstage()
h5_overview(adsimdet)
h5_file=PosixPath('/tmp/docker_ioc/iocad/tmp/test_image_0001.h5')
('/entry/data' in NeXus_data)=True
('/entry/data/dark' in NeXus_data)=True
('/entry/data/data' in NeXus_data)=True
('/entry/data/white' in NeXus_data)=True
('/entry/instrument' in NeXus_data)=True
('/entry/instrument/detector' in NeXus_data)=True
('/entry/instrument/detector/dark' in NeXus_data)=True
('/entry/instrument/detector/data' in NeXus_data)=True
('/entry/instrument/detector/white' in NeXus_data)=True
('/entry/instrument/NDAttributes' in NeXus_data)=True
('/entry/instrument/NDAttributes/HDF5FrameLocation' in NeXus_data)=False
ds=<HDF5 dataset "dark": shape (3, 1024, 1024), type "|u1"> ds.shape=(3, 1024, 1024)
ds=<HDF5 dataset "data": shape (6, 1024, 1024), type "|u1"> ds.shape=(6, 1024, 1024)
ds=<HDF5 dataset "white": shape (2, 1024, 1024), type "|u1"> ds.shape=(2, 1024, 1024)
frame_type: PV='ad:cam1:FrameType_RBV'
value=0 (Normal)
choices=('Normal', 'Background', 'FlatField', 'DblCorrelation')
h5_file=PosixPath('/tmp/docker_ioc/iocad/tmp/test_image_0001.h5')
We’re looking for 3 darks, 6 data, and 2 white frames. All the test results should be True.
Use Bluesky to Acquire Image Frames#
Here, we collect image frames with a custom plan using the bluesky RunEngine (RE).
Custom Bluesky Plan#
[20]:
import bluesky.plan_stubs as bps
def frame_set(det, frame_type=0, num_frames=1, sleep=0.25):
frame_name = "image background white".split()[frame_type]
print(f"{frame_type=} {frame_name=} {num_frames=}")
yield from bps.mv(
det.cam.frame_type, frame_type,
det.cam.num_images, num_frames,
)
if sleep > 0:
yield from bps.sleep(sleep)
yield from bps.mv(det.cam.acquire, 1) # waits for acquire=0
while det.cam.acquire_busy.get(use_monitor=False) != 0:
yield from bps.sleep(0.01)
def series(det, sequence, sleep=1):
total = sum([item[1] for item in sequence])
print("total frames:", total)
print("setup")
yield from bps.mv(
det.hdf1.auto_save, "Yes",
det.hdf1.num_capture, 0,
det.cam.image_mode, "Multiple",
)
yield from bps.mv(det.hdf1.file_write_mode, 'Stream') # TODO: or Capture
yield from bps.stage(det)
for frame_specification in sequence:
frame_type, num_frames = frame_specification
yield from frame_set(det, frame_type, num_frames, sleep=sleep)
yield from bps.mv(det.hdf1.capture, 0) # before file_write_mode = 'Single
yield from bps.unstage(det)
yield from bps.mv(
det.cam.frame_type, IMAGE,
det.hdf1.file_write_mode, 'Single',
)
Acquire Image Frames with Bluesky#
Describe a sequence of acquisitions with different frame types and number of frames to collect with each.
Run this sequence with the custom plan (in the RunEngine), then print an overview of the HDF5 file.
[21]:
SEQUENCE = [
(DARK, 1),
(WHITE, 1),
(IMAGE, 1),
(IMAGE, 1),
(DARK, 1),
(IMAGE, 2),
(WHITE, 1),
(DARK, 1),
]
RE(series(adsimdet, SEQUENCE, sleep=0.0))
h5_overview(adsimdet)
total frames: 9
setup
frame_type=1 frame_name='background' num_frames=1
frame_type=2 frame_name='white' num_frames=1
frame_type=0 frame_name='image' num_frames=1
frame_type=0 frame_name='image' num_frames=1
frame_type=1 frame_name='background' num_frames=1
frame_type=0 frame_name='image' num_frames=2
frame_type=2 frame_name='white' num_frames=1
frame_type=1 frame_name='background' num_frames=1
h5_file=PosixPath('/tmp/docker_ioc/iocad/tmp/test_image_0002.h5')
('/entry/data' in NeXus_data)=True
('/entry/data/dark' in NeXus_data)=True
('/entry/data/data' in NeXus_data)=True
('/entry/data/white' in NeXus_data)=True
('/entry/instrument' in NeXus_data)=True
('/entry/instrument/detector' in NeXus_data)=True
('/entry/instrument/detector/dark' in NeXus_data)=True
('/entry/instrument/detector/data' in NeXus_data)=True
('/entry/instrument/detector/white' in NeXus_data)=True
('/entry/instrument/NDAttributes' in NeXus_data)=True
('/entry/instrument/NDAttributes/HDF5FrameLocation' in NeXus_data)=True
ds=<HDF5 dataset "dark": shape (3, 1024, 1024), type "|u1"> ds.shape=(3, 1024, 1024)
ds=<HDF5 dataset "data": shape (4, 1024, 1024), type "|u1"> ds.shape=(4, 1024, 1024)
ds=<HDF5 dataset "white": shape (2, 1024, 1024), type "|u1"> ds.shape=(2, 1024, 1024)
frame_type: PV='ad:cam1:FrameType_RBV'
value=0 (Normal)
choices=('Normal', 'Background', 'FlatField', 'DblCorrelation')
h5_file=PosixPath('/tmp/docker_ioc/iocad/tmp/test_image_0002.h5')
We are looking for this structure in the HDF5 data file (other structure has been omitted, for clarity):
entry:NXentry
data:NXdata
dark:NX_UINT8[3,1024,1024] = [ ... ]
data:NX_UINT8[4,1024,1024] = [ ... ]
white:NX_UINT8[2,1024,1024] = [ ... ]
instrument:NXinstrument
NDAttributes:NXcollection
HDF5FrameLocation:NX_CHAR[256] = /entry/data/data
detector:NXdetector
dark --> /entry/data/dark
data --> /entry/data/data
white --> /entry/data/white
All the test results should be True.