{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "69428f0e",
"metadata": {},
"source": [
"# The `instrument` package for Bluesky Data Acquisition\n",
"\n",
"From *APS Python Training for Bluesky Data Acquisition*.\n",
"\n",
"**Objective**\n",
"\n",
"In this notebook, we describe the `instrument` package for a simulated X-ray instrument at a user facility such as the Advanced Photon Source. The goal is to use EPICS as much as possible to provide the control system features and to use Bluesky (providing the data acquisition framework) as a thin layer on top of EPICS.\n",
"\n",
"**Overview**\n",
"\n",
"This notebook uses two EPICS servers (IOCs), one, with prefix `\"gp:\"`, providing (simulated) general beamline equipment such as motors, slits, shutter, counters, ... The other IOC, with prefix `\"ad:\"`, provides a simulated EPICS area detector. This notebook will acquaint you with some of the features provided by our Python `instrument` package which uses these IOCs.\n",
"\n",
"\n",
"IOC details
\n",
"\n",
"This simulated instrument is provided using [docker](https://www.docker.com/) images\n",
"for [EPICS base](https://www.aps.anl.gov/epics), the [synApps](https://www.aps.anl.gov/BCDA/synApps) [xxx](https://github.com/epics-modules/xxx) module, and [EPICS area detector](https://areadetector.github.io/master/index.html) [ADSimDetector](https://areadetector.github.io/master/ADSimDetector/simDetector.html?highlight=adsimdetector). The images are based on these software versions:\n",
"\n",
"* Debian GNU/Linux 11 (bullseye)\n",
"* EPICS base 7.0.5\n",
"* synApps 6.2.1\n",
"* area detector 3.11\n",
"\n",
"Two EPICS IOCs are provided:\n",
"\n",
"prefix | description | docker image | documentation\n",
":--- | :--- | :--- | :---\n",
"`ad:` | area detector IOC | [prjemian/custom-synapps-6.2-ad-3.10](https://hub.docker.com/r/prjemian/custom-synapps-6.2-ad-3.10) | https://github.com/prjemian/epics-docker/tree/main/v1.1/n6_custom_areaDetector\n",
"`gp:` | general purpose IOC | [prjemian/prjemian/custom-synapps-6.2](https://hub.docker.com/r/prjemian/custom-synapps-6.2) | https://github.com/prjemian/epics-docker/tree/main/v1.1/n5_custom_synApps\n",
"\n",
" "
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "8adfd058",
"metadata": {},
"source": [
"## Start the `instrument` package\n",
"\n",
"To start the `instrument`, use this Python startup [snippet](https://bcda-aps.github.io/bluesky_training/reference/_ipython.html#python-snippet-to-start-instrument-for-data-collection):"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "f25dae9d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/home/prjemian/bluesky/instrument/_iconfig.py\n",
"Activating auto-logging. Current session state plus future input saved.\n",
"Filename : /home/prjemian/Documents/projects/BCDA-APS/bluesky_training/docs/source/instrument/.logs/ipython_console.log\n",
"Mode : rotate\n",
"Output logging : True\n",
"Raw input log : False\n",
"Timestamping : True\n",
"State : active\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"I Fri-11:15:53 - ############################################################ startup\n",
"I Fri-11:15:53 - logging started\n",
"I Fri-11:15:53 - logging level = 10\n",
"I Fri-11:15:53 - /home/prjemian/bluesky/instrument/session_logs.py\n",
"I Fri-11:15:53 - /home/prjemian/bluesky/instrument/collection.py\n",
"I Fri-11:15:53 - CONDA_PREFIX = /opt/miniconda3\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Exception reporting mode: Minimal\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"I Fri-11:15:53 - xmode exception level: 'Minimal'\n",
"I Fri-11:15:53 - /home/prjemian/bluesky/instrument/mpl/notebook.py\n",
"I Fri-11:15:53 - #### Bluesky Framework ####\n",
"I Fri-11:15:53 - /home/prjemian/bluesky/instrument/framework/check_python.py\n",
"I Fri-11:15:53 - /home/prjemian/bluesky/instrument/framework/check_bluesky.py\n",
"I Fri-11:15:53 - /home/prjemian/bluesky/instrument/framework/initialize.py\n",
"I Fri-11:15:53 - RunEngine metadata saved in directory: /home/prjemian/Bluesky_RunEngine_md\n",
"I Fri-11:15:53 - using databroker catalog 'training'\n",
"I Fri-11:15:53 - using ophyd control layer: pyepics\n",
"I Fri-11:15:53 - /home/prjemian/bluesky/instrument/framework/metadata.py\n",
"I Fri-11:15:53 - /home/prjemian/bluesky/instrument/epics_signal_config.py\n",
"I Fri-11:15:53 - Using RunEngine metadata for scan_id\n",
"I Fri-11:15:54 - #### Devices ####\n",
"I Fri-11:15:54 - /home/prjemian/bluesky/instrument/devices/area_detector.py\n",
"I Fri-11:15:54 - /home/prjemian/bluesky/instrument/devices/calculation_records.py\n",
"I Fri-11:15:56 - /home/prjemian/bluesky/instrument/devices/fourc_diffractometer.py\n",
"I Fri-11:15:56 - /home/prjemian/bluesky/instrument/devices/ioc_stats.py\n",
"I Fri-11:15:56 - /home/prjemian/bluesky/instrument/devices/kohzu_monochromator.py\n",
"I Fri-11:15:56 - /home/prjemian/bluesky/instrument/devices/motors.py\n",
"I Fri-11:15:56 - /home/prjemian/bluesky/instrument/devices/noisy_detector.py\n",
"I Fri-11:15:56 - /home/prjemian/bluesky/instrument/devices/scaler.py\n",
"I Fri-11:15:57 - /home/prjemian/bluesky/instrument/devices/shutter_simulator.py\n",
"I Fri-11:15:57 - /home/prjemian/bluesky/instrument/devices/simulated_fourc.py\n",
"I Fri-11:15:57 - /home/prjemian/bluesky/instrument/devices/simulated_kappa.py\n",
"I Fri-11:15:57 - /home/prjemian/bluesky/instrument/devices/slits.py\n",
"I Fri-11:15:57 - /home/prjemian/bluesky/instrument/devices/sixc_diffractometer.py\n",
"I Fri-11:15:58 - /home/prjemian/bluesky/instrument/devices/temperature_signal.py\n",
"I Fri-11:15:58 - #### Callbacks ####\n",
"I Fri-11:15:58 - /home/prjemian/bluesky/instrument/callbacks/spec_data_file_writer.py\n",
"I Fri-11:15:58 - #### Plans ####\n",
"I Fri-11:15:58 - /home/prjemian/bluesky/instrument/plans/lup_plan.py\n",
"I Fri-11:15:58 - /home/prjemian/bluesky/instrument/plans/peak_finder_example.py\n",
"I Fri-11:15:58 - /home/prjemian/bluesky/instrument/utils/image_analysis.py\n",
"I Fri-11:15:58 - #### Utilities ####\n",
"I Fri-11:15:58 - writing to SPEC file: /home/prjemian/Documents/projects/BCDA-APS/bluesky_training/docs/source/instrument/20230414-111558.dat\n",
"I Fri-11:15:58 - >>>> Using default SPEC file name <<<<\n",
"I Fri-11:15:58 - file will be created when bluesky ends its next scan\n",
"I Fri-11:15:58 - to change SPEC file, use command: newSpecFile('title')\n",
"I Fri-11:15:58 - #### Startup is complete. ####\n"
]
}
],
"source": [
"import pathlib, sys\n",
"sys.path.append(str(pathlib.Path.home() / \"bluesky\"))\n",
"from instrument.collection import *"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "872cd150",
"metadata": {},
"source": [
"## Description\n",
"\n",
"Might be a good idea to know now what this instrument package provides. \n",
"\n",
"Notably, the table includes:\n",
"\n",
"ophyd name(s) | Description | label(s)\n",
":--- | :---| :---\n",
"`adsimdet` | simulated EPICS area detector | `area_detector`\n",
"`calcs` & `calcouts` | calculation support |\n",
"`gp_stats` | details about the general purpose IOC |\n",
"`I0`, `diode`, ... | named scaler channels | `counter`\n",
"`iconfig` | instrument configuration parameters\n",
"`m1` .. `m16` | 16 simulated EPICS motors | `motor`\n",
"`noisy` | simulated diffraction peak |\n",
"`scaler1` | simulated 16-channel EPICS scaler | `scalers`\n",
"`shutter` | simulated shutter | \n",
"`temperature` | simulated temperature controller |"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "096aca61",
"metadata": {},
"source": [
"### `iconfig`\n",
"\n",
"`iconfig`, a Python dictionary, contains the instrument's configuration details defined in one place for the user to control. These details, defined in [instrument/iconfig.yml](https://github.com/BCDA-APS/bluesky_training/blob/main/bluesky/instrument/iconfig.yml), are used in various places in the `instrument` package. Any changes to the `iconfig.yml` file will take effect the _next_ time the instrument is started."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2f091ec0",
"metadata": {},
"source": [
"### `adsimdet`"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "ff54948c",
"metadata": {},
"source": [
"The EPICS Area Detector ADSimDetector is included with this instrument as `adsimdet`. The detector is a monochrome 1k x 1k frame. The image is written to an HDF5 file and made available to the [databroker](#databroker) via a shared volume from the docker container. `adsimdet` can be used as a detector.\n",
"\n",
"The image is a simulated diffraction spot with center randomly-chosen between 100..900 on both axes. The width is also random as is the peak intensity and noise level. Furthermore, to simulate realistic conditions, the center position of both axes is adjusted by a few pixels using a pair of *swait* records (`gp:userCalc9` & `gp:userCalc10`) updating from random numbers."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "7d735ebd",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"data keys (* hints)\n",
"-------------------\n",
"\n",
"read attrs\n",
"----------\n",
"hdf1 MyHDF5Plugin ('adsimdet_hdf1')\n",
"\n",
"config keys\n",
"-----------\n",
"adsimdet_cam_acquire_period\n",
"adsimdet_cam_acquire_time\n",
"adsimdet_cam_image_mode\n",
"adsimdet_cam_manufacturer\n",
"adsimdet_cam_model\n",
"adsimdet_cam_num_exposures\n",
"adsimdet_cam_num_images\n",
"adsimdet_cam_trigger_mode\n",
"\n",
"configuration attrs\n",
"-------------------\n",
"cam SimDetectorCam_V34 ('adsimdet_cam')\n",
"cam.acquire_period EpicsSignalWithRBV ('adsimdet_cam_acquire_period')\n",
"cam.acquire_time EpicsSignalWithRBV ('adsimdet_cam_acquire_time')\n",
"cam.image_mode EpicsSignalWithRBV ('adsimdet_cam_image_mode')\n",
"cam.manufacturer EpicsSignalRO ('adsimdet_cam_manufacturer')\n",
"cam.model EpicsSignalRO ('adsimdet_cam_model')\n",
"cam.num_exposures EpicsSignalWithRBV ('adsimdet_cam_num_exposures')\n",
"cam.num_images EpicsSignalWithRBV ('adsimdet_cam_num_images')\n",
"cam.trigger_mode EpicsSignalWithRBV ('adsimdet_cam_trigger_mode')\n",
"hdf1 MyHDF5Plugin ('adsimdet_hdf1')\n",
"\n",
"unused attrs\n",
"------------\n",
"configuration_names ArrayAttributeSignal('adsimdet_configuration_names')\n",
"image ImagePlugin_V34 ('adsimdet_image')\n",
"pva PvaPlugin_V34 ('adsimdet_pva')\n",
"\n"
]
}
],
"source": [
"adsimdet.summary()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "3c7946eb",
"metadata": {},
"source": [
"### `scaler1`"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c9a1d6e0",
"metadata": {},
"source": [
"A simulated 16-channel scaler is configured with several channels of pulse counters, typical of a synchrotron beamline. `scaler1` can be used as a detector."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "7122a2f9",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"data keys (* hints)\n",
"-------------------\n",
"*I0\n",
"*I00\n",
"*I000\n",
"*roi1\n",
" scaler1_time\n",
"*scint\n",
"*timebase\n",
"\n",
"read attrs\n",
"----------\n",
"channels Channels ('scaler1_channels')\n",
"channels.chan01 ScalerChannel ('scaler1_channels_chan01')\n",
"channels.chan01.s EpicsSignalRO ('timebase')\n",
"channels.chan02 ScalerChannel ('scaler1_channels_chan02')\n",
"channels.chan02.s EpicsSignalRO ('I0')\n",
"channels.chan04 ScalerChannel ('scaler1_channels_chan04')\n",
"channels.chan04.s EpicsSignalRO ('scint')\n",
"channels.chan05 ScalerChannel ('scaler1_channels_chan05')\n",
"channels.chan05.s EpicsSignalRO ('I000')\n",
"channels.chan06 ScalerChannel ('scaler1_channels_chan06')\n",
"channels.chan06.s EpicsSignalRO ('I00')\n",
"channels.chan07 ScalerChannel ('scaler1_channels_chan07')\n",
"channels.chan07.s EpicsSignalRO ('roi1')\n",
"time EpicsSignal ('scaler1_time')\n",
"\n",
"config keys\n",
"-----------\n",
"scaler1_auto_count_delay\n",
"scaler1_auto_count_time\n",
"scaler1_channels_chan01_chname\n",
"scaler1_channels_chan01_gate\n",
"scaler1_channels_chan01_preset\n",
"scaler1_channels_chan02_chname\n",
"scaler1_channels_chan02_gate\n",
"scaler1_channels_chan02_preset\n",
"scaler1_channels_chan04_chname\n",
"scaler1_channels_chan04_gate\n",
"scaler1_channels_chan04_preset\n",
"scaler1_channels_chan05_chname\n",
"scaler1_channels_chan05_gate\n",
"scaler1_channels_chan05_preset\n",
"scaler1_channels_chan06_chname\n",
"scaler1_channels_chan06_gate\n",
"scaler1_channels_chan06_preset\n",
"scaler1_channels_chan07_chname\n",
"scaler1_channels_chan07_gate\n",
"scaler1_channels_chan07_preset\n",
"scaler1_count_mode\n",
"scaler1_delay\n",
"scaler1_egu\n",
"scaler1_freq\n",
"scaler1_preset_time\n",
"\n",
"configuration attrs\n",
"-------------------\n",
"channels Channels ('scaler1_channels')\n",
"channels.chan01 ScalerChannel ('scaler1_channels_chan01')\n",
"channels.chan01.chname EpicsSignal ('scaler1_channels_chan01_chname')\n",
"channels.chan01.preset EpicsSignal ('scaler1_channels_chan01_preset')\n",
"channels.chan01.gate EpicsSignal ('scaler1_channels_chan01_gate')\n",
"channels.chan02 ScalerChannel ('scaler1_channels_chan02')\n",
"channels.chan02.chname EpicsSignal ('scaler1_channels_chan02_chname')\n",
"channels.chan02.preset EpicsSignal ('scaler1_channels_chan02_preset')\n",
"channels.chan02.gate EpicsSignal ('scaler1_channels_chan02_gate')\n",
"channels.chan04 ScalerChannel ('scaler1_channels_chan04')\n",
"channels.chan04.chname EpicsSignal ('scaler1_channels_chan04_chname')\n",
"channels.chan04.preset EpicsSignal ('scaler1_channels_chan04_preset')\n",
"channels.chan04.gate EpicsSignal ('scaler1_channels_chan04_gate')\n",
"channels.chan05 ScalerChannel ('scaler1_channels_chan05')\n",
"channels.chan05.chname EpicsSignal ('scaler1_channels_chan05_chname')\n",
"channels.chan05.preset EpicsSignal ('scaler1_channels_chan05_preset')\n",
"channels.chan05.gate EpicsSignal ('scaler1_channels_chan05_gate')\n",
"channels.chan06 ScalerChannel ('scaler1_channels_chan06')\n",
"channels.chan06.chname EpicsSignal ('scaler1_channels_chan06_chname')\n",
"channels.chan06.preset EpicsSignal ('scaler1_channels_chan06_preset')\n",
"channels.chan06.gate EpicsSignal ('scaler1_channels_chan06_gate')\n",
"channels.chan07 ScalerChannel ('scaler1_channels_chan07')\n",
"channels.chan07.chname EpicsSignal ('scaler1_channels_chan07_chname')\n",
"channels.chan07.preset EpicsSignal ('scaler1_channels_chan07_preset')\n",
"channels.chan07.gate EpicsSignal ('scaler1_channels_chan07_gate')\n",
"count_mode EpicsSignal ('scaler1_count_mode')\n",
"delay EpicsSignal ('scaler1_delay')\n",
"auto_count_delay EpicsSignal ('scaler1_auto_count_delay')\n",
"freq EpicsSignal ('scaler1_freq')\n",
"preset_time EpicsSignal ('scaler1_preset_time')\n",
"auto_count_time EpicsSignal ('scaler1_auto_count_time')\n",
"egu EpicsSignal ('scaler1_egu')\n",
"\n",
"unused attrs\n",
"------------\n",
"count EpicsSignal ('scaler1_count')\n",
"update_rate EpicsSignal ('scaler1_update_rate')\n",
"auto_count_update_rate EpicsSignal ('scaler1_auto_count_update_rate')\n",
"\n"
]
}
],
"source": [
"scaler1.summary()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "7c9ddd29",
"metadata": {},
"source": [
"The timebase is always shown. The additional channels (using names from the scaler GUI screen) are shown output of `listdevice(scaler1)` below.\n",
"\n",
"Note that in the instrument package, the channel names were assigned on startup. An operating beamline would not do define the names here but instead, let the instrument team name these channels in the GUI screen at the time the cables from the pulse detectors are connected physically to the scaler.\n",
"\n",
"Count the scaler from the command line:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "cae26813",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[This data will not be saved. Use the RunEngine to collect data.]\n",
"timebase 16000000.0\n",
"I0 8.0\n",
"scint 7.0\n",
"I000 6.0\n",
"I00 8.0\n",
"roi1 6.0\n",
"scaler1_time 1.6\n"
]
}
],
"source": [
"%ct scalers"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "65060e52",
"metadata": {},
"source": [
"NOTE: the actual counts are random numbers. Also, the EPICS soft scaler seems to add 0.1s to the counting time. Might be a bug with the EPICS scaler code."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "23ffed2c",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"============================== ========== ==========================\n",
"data name value timestamp \n",
"============================== ========== ==========================\n",
"scaler1_channels_chan01_chname timebase 2023-04-14 10:58:36.645292\n",
"timebase 16000000.0 2023-04-14 11:16:00.339417\n",
"scaler1_channels_chan01_preset 15000000.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan01_gate Y 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan02_chname I0 2023-04-14 10:58:36.645292\n",
"I0 8.0 2023-04-14 11:16:00.339417\n",
"scaler1_channels_chan02_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan02_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan03_chname scint 2023-04-14 10:58:36.645292\n",
"scint 7.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan03_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan03_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan04_chname scint 2023-04-14 10:58:36.645292\n",
"scint 7.0 2023-04-14 11:16:00.339417\n",
"scaler1_channels_chan04_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan04_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan05_chname I000 2023-04-14 10:58:36.645292\n",
"I000 6.0 2023-04-14 11:16:00.339417\n",
"scaler1_channels_chan05_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan05_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan06_chname I00 2023-04-14 10:58:36.645292\n",
"I00 8.0 2023-04-14 11:16:00.339417\n",
"scaler1_channels_chan06_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan06_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan07_chname roi1 2023-04-14 10:58:36.645292\n",
"roi1 6.0 2023-04-14 11:16:00.339417\n",
"scaler1_channels_chan07_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan07_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan08_chname 2023-04-14 10:58:36.645292\n",
" 8.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan08_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan08_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan09_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan09_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan09_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan10_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan10_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan10_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan11_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan11_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan11_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan12_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan12_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan12_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan13_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan13_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan13_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan14_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan14_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan14_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan15_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan15_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan15_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan16_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan16_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan16_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan17_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan17_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan17_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan18_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan18_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan18_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan19_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan19_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan19_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan20_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan20_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan20_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan21_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan21_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan21_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan22_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan22_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan22_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan23_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan23_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan23_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan24_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan24_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan24_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan25_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan25_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan25_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan26_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan26_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan26_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan27_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan27_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan27_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan28_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan28_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan28_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan29_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan29_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan29_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan30_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan30_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan30_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan31_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan31_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan31_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan32_chname 2023-04-14 10:58:36.645292\n",
" 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan32_preset 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_channels_chan32_gate N 2023-04-14 10:58:36.645292\n",
"scaler1_count 0 2023-04-14 10:58:36.645292\n",
"scaler1_count_mode OneShot 2023-04-14 10:58:36.645292\n",
"scaler1_delay 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_auto_count_delay 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_time 1.6 2023-04-14 10:58:36.645292\n",
"scaler1_freq 10000000.0 2023-04-14 10:58:36.645292\n",
"scaler1_preset_time 1.5 2023-04-14 10:58:36.645292\n",
"scaler1_auto_count_time 1.0 2023-04-14 10:58:36.645292\n",
"scaler1_update_rate 10.0 2023-04-14 10:58:36.645292\n",
"scaler1_auto_count_update_rate 0.0 2023-04-14 10:58:36.645292\n",
"scaler1_egu 2023-04-14 10:58:36.645292\n",
"============================== ========== =========================="
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"listdevice(scaler1)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "400f3433",
"metadata": {},
"source": [
"As an added convenience, shortcut names for these channels were also assigned to local Python symbols (since the object path to the channels is easy to forget) in `instrument/devices/scaler.py` with these lines:\n",
"\n",
"```python\n",
"timebase = scaler1.channels.chan01.s\n",
"I0 = scaler1.channels.chan02.s\n",
"scint = scaler1.channels.chan03.s\n",
"diode = scaler1.channels.chan04.s\n",
"```"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "b032df33",
"metadata": {},
"source": [
"This command: `scaler1.select_channels()` configures `scaler` to use just the channels with non-empty names. To limit data collection to a subset of these channels, name the channels in a list argument, such as: `scaler1.select_channels(['I0', 'diode'])`"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "9a6f9313",
"metadata": {},
"source": [
"### `temperature`"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4a82dc30",
"metadata": {},
"source": [
"The temperature simulator, implemented in `gp:userCalc8` works like a temperature controller, with setpoint and readback values. `temperature` can be used as a detector **and** as a positioner (like a motor).\n",
"\n",
"Additional controls for noise, update interval, maximum change per update, in-position tolerance, and done are provided, either through EPICS or in Python. Once a new setpoint is entered, the readback will progress towards it, limited by the max change and the update interval. Once at the setpoint, noise continues to be added to the temperature to provide realism."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "23b8af86",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"data keys (* hints)\n",
"-------------------\n",
"*temperature\n",
" temperature_calculation\n",
" temperature_description\n",
" temperature_done\n",
" temperature_max_change\n",
" temperature_noise\n",
" temperature_previous_value_pv\n",
" temperature_scanning_rate\n",
" temperature_setpoint\n",
" temperature_tolerance\n",
"\n",
"read attrs\n",
"----------\n",
"setpoint EpicsSignal ('temperature_setpoint')\n",
"readback EpicsSignal ('temperature')\n",
"done Signal ('temperature_done')\n",
"calculation EpicsSignal ('temperature_calculation')\n",
"description EpicsSignal ('temperature_description')\n",
"max_change EpicsSignal ('temperature_max_change')\n",
"noise EpicsSignal ('temperature_noise')\n",
"previous_value_pv EpicsSignal ('temperature_previous_value_pv')\n",
"scanning_rate EpicsSignal ('temperature_scanning_rate')\n",
"tolerance EpicsSignal ('temperature_tolerance')\n",
"\n",
"config keys\n",
"-----------\n",
"\n",
"configuration attrs\n",
"-------------------\n",
"\n",
"unused attrs\n",
"------------\n",
"report_dmov_changes Signal ('temperature_report_dmov_changes')\n",
"\n"
]
},
{
"data": {
"text/plain": [
"=============================== =================================== ==========================\n",
"data name value timestamp \n",
"=============================== =================================== ==========================\n",
"temperature_setpoint 25.0 2023-04-14 11:15:58.067527\n",
"temperature 25.07474631876097 2023-04-14 11:16:00.268549\n",
"temperature_done True 2023-04-14 11:16:00.274526\n",
"temperature_calculation A+max(-D,min(D,(B-A)))+C*(RNDM-0.5) 2023-04-14 11:15:58.071359\n",
"temperature_description temperature 2023-04-14 11:15:58.065403\n",
"temperature_max_change 2.0 2023-04-14 11:15:58.070733\n",
"temperature_noise 1.0 2023-04-14 11:15:58.070030\n",
"temperature_previous_value_pv gp:userCalc8.VAL 2023-04-14 11:15:58.065403\n",
"temperature_scanning_rate 5 2023-04-14 11:15:58.071359\n",
"temperature_tolerance 1.0 2023-04-14 11:15:58.071359\n",
"temperature_report_dmov_changes False 2023-04-14 11:15:58.066705\n",
"=============================== =================================== =========================="
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"temperature.summary()\n",
"listdevice(temperature)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "8fb5a522",
"metadata": {},
"source": [
"### motors"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "26a5e7a4",
"metadata": {},
"source": [
"There are 16 soft motor channels: `gp:m1` .. `gp:m16`. Any motor can be used as a detector **and** as a positioner.\n",
"\n",
"The first motor, `gp:m1`, is used to compute a simulated 1-D diffraction peak ([noisy](#noisy)) using `gp:userCalc1`.\n",
"\n",
"To make it easier to change the motor step size, a custom `MyEpicsMotor` class was made, subclassing from `ophyd.EpicsMotor` and adding a component to access the motor's `.SREV` (steps/revolution) field. The default motor step size is 0.01 (with SREV=200). For only `m1`, the step size has been changed to 0.001 by setting SREV=2000.\n",
"\n",
"NOTE: Changing SREV (in this simulator* seems to affect the soft motor's simulated hardware limit. Be careful if you adjust SREV to higher numbers (smaller step sizes) that the expected range of motion remains sufficient for your measurements. Again, another thing to change in the EPICS support."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "888b9a30",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"data keys (* hints)\n",
"-------------------\n",
"*m1\n",
" m1_user_setpoint\n",
"\n",
"read attrs\n",
"----------\n",
"user_readback EpicsSignalRO ('m1')\n",
"user_setpoint EpicsSignal ('m1_user_setpoint')\n",
"\n",
"config keys\n",
"-----------\n",
"m1_acceleration\n",
"m1_motor_egu\n",
"m1_user_offset\n",
"m1_user_offset_dir\n",
"m1_velocity\n",
"\n",
"configuration attrs\n",
"-------------------\n",
"user_offset EpicsSignal ('m1_user_offset')\n",
"user_offset_dir EpicsSignal ('m1_user_offset_dir')\n",
"velocity EpicsSignal ('m1_velocity')\n",
"acceleration EpicsSignal ('m1_acceleration')\n",
"motor_egu EpicsSignal ('m1_motor_egu')\n",
"\n",
"unused attrs\n",
"------------\n",
"offset_freeze_switch EpicsSignal ('m1_offset_freeze_switch')\n",
"set_use_switch EpicsSignal ('m1_set_use_switch')\n",
"motor_is_moving EpicsSignalRO ('m1_motor_is_moving')\n",
"motor_done_move EpicsSignalRO ('m1_motor_done_move')\n",
"high_limit_switch EpicsSignalRO ('m1_high_limit_switch')\n",
"low_limit_switch EpicsSignalRO ('m1_low_limit_switch')\n",
"high_limit_travel EpicsSignal ('m1_high_limit_travel')\n",
"low_limit_travel EpicsSignal ('m1_low_limit_travel')\n",
"direction_of_travel EpicsSignal ('m1_direction_of_travel')\n",
"motor_stop EpicsSignal ('m1_motor_stop')\n",
"home_forward EpicsSignal ('m1_home_forward')\n",
"home_reverse EpicsSignal ('m1_home_reverse')\n",
"steps_per_revolution EpicsSignal ('m1_steps_per_revolution')\n",
"\n"
]
},
{
"data": {
"text/plain": [
"======================= ======= ==========================\n",
"data name value timestamp \n",
"======================= ======= ==========================\n",
"m1 0.0 2023-04-14 10:58:33.041083\n",
"m1_user_setpoint 0.0 2023-04-14 10:58:33.041083\n",
"m1_user_offset 0.0 2023-04-14 10:58:33.041083\n",
"m1_user_offset_dir 0 2023-04-14 10:58:33.041083\n",
"m1_offset_freeze_switch 0 2023-04-14 10:58:33.041083\n",
"m1_set_use_switch 0 2023-04-14 10:58:33.041083\n",
"m1_velocity 1.0 2023-04-14 10:58:33.041083\n",
"m1_acceleration 0.2 2023-04-14 10:58:33.041083\n",
"m1_motor_egu degrees 2023-04-14 10:58:33.041083\n",
"m1_motor_is_moving 0 2023-04-14 10:58:33.041083\n",
"m1_motor_done_move 1 2023-04-14 10:58:33.041083\n",
"m1_high_limit_switch 0 2023-04-14 10:58:33.041083\n",
"m1_low_limit_switch 0 2023-04-14 10:58:33.041083\n",
"m1_high_limit_travel 1000.0 2023-04-14 10:58:33.041083\n",
"m1_low_limit_travel -1000.0 2023-04-14 10:58:33.041083\n",
"m1_direction_of_travel 0 2023-04-14 10:58:33.041083\n",
"m1_motor_stop 0 2023-04-14 10:58:33.041083\n",
"m1_home_forward 0 2023-04-14 10:58:33.041083\n",
"m1_home_reverse 0 2023-04-14 10:58:33.041083\n",
"m1_steps_per_revolution 2000 2023-04-14 10:58:33.041083\n",
"======================= ======= =========================="
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m1.summary()\n",
"listdevice(m1)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "b7d3f5ad",
"metadata": {},
"source": [
"### `noisy`"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "79649ad2",
"metadata": {},
"source": [
"A simulated diffraction peak is computed in 1-D using a Lorentzian function in EPICS PV `gp:userCalc1`. The simulation uses the position of motor `gp:m1` and random choices for center, width, noise, and peak intensity (scale factor). `noisy` can be used as a detector.\n",
"\n",
"NOTE: `noisy` is an `ophyd.EpicsSignal` (variant), thus it lacks the `.summary()` method that `ophyd.Device` objects have."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "d939e068",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"========= ================= ==========================\n",
"data name value timestamp \n",
"========= ================= ==========================\n",
"noisy 685.3703376927396 2023-04-14 11:15:56.876793\n",
"========= ================= =========================="
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"listdevice(noisy)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "b911b8b3",
"metadata": {},
"source": [
"### plans\n",
"\n",
"Several examples of user plan are provided in file `instrument/plans/peak_finder_example.py`. These plans are described and used in the notebook [Lineup a 1-D peak](https://nbviewer.jupyter.org/github/BCDA-APS/bluesky_instrument_training/blob/main/lineup_1d_peak.ipynb)."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "3ef17dbb",
"metadata": {},
"source": [
"## Log files\n",
"\n",
"For diagnostic and general support, log files are created to record activity. In the working directory, the log files are written to a `./.logs` subdirectory.\n",
"\n",
"There are two kinds of file, one that records user commands and the python\n",
"result, the other records items sent to the Python\n",
"[logging](https://docs.python.org/3/library/logging.html) package.\n",
"\n",
"In the IPython session, use the `!` to run a linux command:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "080a4be9",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"total 7.5M\n",
"-rw-rw-r-- 1 prjemian 15K Apr 14 11:16 ipython_console.log\n",
"-rw-rw-r-- 1 prjemian 15K Apr 14 10:58 ipython_console.log.001~\n",
"-rw-rw-r-- 1 prjemian 16K Apr 14 10:57 ipython_console.log.002~\n",
"-rw-rw-r-- 1 prjemian 5.8K Apr 14 11:15 ipython_logger.log\n",
"-rw-rw-r-- 1 prjemian 455K Apr 14 11:16 ophyd.control_layer.log\n",
"-rw-rw-r-- 1 prjemian 1.0M Apr 14 11:15 ophyd.control_layer.log.1\n",
"-rw-rw-r-- 1 prjemian 1.0M Apr 14 11:12 ophyd.control_layer.log.2\n",
"-rw-rw-r-- 1 prjemian 1.0M Apr 14 11:06 ophyd.control_layer.log.3\n",
"-rw-rw-r-- 1 prjemian 1.0M Apr 14 11:01 ophyd.control_layer.log.4\n",
"-rw-rw-r-- 1 prjemian 1.0M Apr 14 10:58 ophyd.control_layer.log.5\n",
"-rw-rw-r-- 1 prjemian 1.0M Apr 14 10:55 ophyd.control_layer.log.6\n",
"-rw-rw-r-- 1 prjemian 1.0M Apr 14 10:49 ophyd.control_layer.log.7\n"
]
}
],
"source": [
"!ls -lAFgh .logs"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "6bbeff1a",
"metadata": {},
"source": [
"Let's take a look at a few lines of each type of file, to get a feel for the information logged.\n",
"\n",
"The `ipython_console.log` file is created for every session (every time IPython is started or every time the Jupyter kernel is started). It records the commands that were entered and the output, if any, from each command. It does not record anything that was sent to the console by `print()` statements. The older log files are numbered, higher number is older (each new session shifts these numbers by 1)."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "0e8046f8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# IPython log file\n",
"\n",
"# Fri, 14 Apr 2023 11:15:58\n",
"adsimdet.summary()\n",
"# Fri, 14 Apr 2023 11:15:58\n",
"scaler1.summary()\n",
"# Fri, 14 Apr 2023 11:15:58\n",
"get_ipython().run_line_magic('ct', 'scalers')\n",
"# Fri, 14 Apr 2023 11:16:00\n",
"listdevice(scaler1)\n"
]
}
],
"source": [
"!head .logs/ipython_console.log"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "bc5568e3",
"metadata": {},
"source": [
"The `ipython_logger.log` file(s) contain the output from calls to the `logger`. The files are appended with new logger reports until the file reaches ca. 1 MB. (Larger files are slow to append.) Then the file is given a number and a new logger file is created. Higher numbers are older. At most, 9 numbered files are retained (to avoid filling disk storage with unnecessary diagnostics).\n",
"\n",
"A single *logger* file may contain reports from several sessions. The third piece of information is the `pid` (process identifier) of the session. The PID is assigned by the operating system when the session is started.\n",
"\n",
"The logger file contains a full report while the report's representation is more terse. (Users do not always want the full details. Just remember they are available in the logger file.) Here's the last few logger lines as shown on the console:\n",
"\n",
" I Wed-00:24:52 - /home/mintadmin/Documents/projects/BCDA-APS/bluesky_instrument_training/instrument/devices/temperature_signal.py\n",
" I Wed-00:24:52 - /home/mintadmin/Documents/projects/BCDA-APS/bluesky_instrument_training/instrument/plans/peak_finder_example.py\n",
" I Wed-00:24:52 - Startup is complete.\n",
"\n",
"Those same lines appear in the logger file as:\n",
"\n",
" |2021-02-24 00:24:52.091|INFO|25986|bluesky-session|temperature_signal|11|MainThread| - /home/mintadmin/Documents/projects/BCDA-APS/bluesky_instrument_training/instrument/devices/temperature_signal.py\n",
" |2021-02-24 00:24:52.165|INFO|25986|bluesky-session|peak_finder_example|15|MainThread| - /home/mintadmin/Documents/projects/BCDA-APS/bluesky_instrument_training/instrument/plans/peak_finder_example.py\n",
" |2021-02-24 00:24:52.179|INFO|25986|bluesky-session|collection|23|MainThread| - Startup is complete.\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "a739acbc",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"|2023-04-14 11:15:53.916|INFO|3701168|bluesky-session|collection|29|MainThread| - #### Bluesky Framework ####\n",
"|2023-04-14 11:15:54.025|INFO|3701168|bluesky-session|collection|32|MainThread| - #### Devices ####\n",
"|2023-04-14 11:15:58.072|INFO|3701168|bluesky-session|collection|35|MainThread| - #### Callbacks ####\n",
"|2023-04-14 11:15:58.079|INFO|3701168|bluesky-session|collection|38|MainThread| - #### Plans ####\n",
"|2023-04-14 11:15:58.130|INFO|3701168|bluesky-session|collection|41|MainThread| - #### Utilities ####\n",
"|2023-04-14 11:15:58.131|INFO|3701168|bluesky-session|collection|49|MainThread| - writing to SPEC file: /home/prjemian/Documents/projects/BCDA-APS/bluesky_training/docs/source/instrument/20230414-111558.dat\n",
"|2023-04-14 11:15:58.133|INFO|3701168|bluesky-session|collection|50|MainThread| - >>>> Using default SPEC file name <<<<\n",
"|2023-04-14 11:15:58.136|INFO|3701168|bluesky-session|collection|51|MainThread| - file will be created when bluesky ends its next scan\n",
"|2023-04-14 11:15:58.137|INFO|3701168|bluesky-session|collection|52|MainThread| - to change SPEC file, use command: newSpecFile('title')\n",
"|2023-04-14 11:15:58.138|INFO|3701168|bluesky-session|collection|56|MainThread| - #### Startup is complete. ####\n"
]
}
],
"source": [
"!tail .logs/ipython_logger.log"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c2a49bbd",
"metadata": {},
"source": [
"## SPEC data files\n",
"\n",
"A [SPEC](https://certif.com) data file records a copy of the scan data (area detector images are not stored in SPEC files). By default, the file is created in the present working directory using a name constructed from the date and time with a `.dat` file extension. Unlike SPEC, new data is written *at the end of a scan*. It is also possible to use the [SpecWriterCallback](https://apstools.readthedocs.io/en/latest/source/_filewriters.html#apstools.filewriters.SpecWriterCallback) to write data extracted from the database after the experiment is done.\n",
"\n",
"The 2021-03 training does not cover the use of these files. You are free to examine them yourselves.\n",
"\n",
"Example SPEC data file from a bluesky session:\n",
"\n",
"\n",
"\n",
" #F /home/mintadmin/Documents/projects/BCDA-APS/bluesky_training/20210723-205451.dat\n",
" #E 1627091691\n",
" #D Fri Jul 23 20:54:51 2021\n",
" #C Bluesky user = mintadmin host = mint-vm\n",
" #O0 \n",
" #o0 \n",
"\n",
" #S 344 scan(detectors=['noisy', 'th_tth_permit'], num=11, args='['theta', 4.5, 3.0, 'ttheta', 9, 6]', per_step='None')\n",
" #D Fri Jul 23 20:55:57 2021\n",
" #C Fri Jul 23 20:55:57 2021. plan_type = generator\n",
" #C Fri Jul 23 20:55:57 2021. uid = 17ebe732-8cff-458e-8ebd-f7c5207b2e46\n",
" #MD uid = 17ebe732-8cff-458e-8ebd-f7c5207b2e46\n",
" #MD beamline_id = Bluesky_training\n",
" #MD detectors = ['noisy', 'th_tth_permit']\n",
" #MD instrument_name = Bluesky Case Studies\n",
" #MD login_id = mintadmin@mint-vm\n",
" #MD motors = ('theta', 'ttheta')\n",
" #MD notebook = UB_autosave\n",
" #MD num_intervals = 10\n",
" #MD num_points = 11\n",
" #MD objective = Demonstrate UB matrix save & restore\n",
" #MD pid = 10389\n",
" #MD plan_pattern = inner_product\n",
" #MD plan_pattern_args = {'num': 11, 'args': [\"EpicsMotor(prefix='gp:m1', name='theta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 4.5, 3.0, \"EpicsMotor(prefix='gp:m2', name='ttheta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 9, 6]}\n",
" #MD plan_pattern_module = bluesky.plan_patterns\n",
" #MD proposal_id = training\n",
" #MD versions = {'apstools': '1.5.1', 'bluesky': '1.7.0', 'databroker': '1.2.3', 'epics': '3.5.0', 'h5py': '3.2.1', 'intake': '0.6.2', 'matplotlib': '3.3.4', 'numpy': '1.20.3', 'ophyd': '1.6.1', 'pyRestTable': '2020.0.3', 'spec2nexus': '2021.1.8'}\n",
" #P0 \n",
" #N 8\n",
" #L theta ttheta Epoch_float Epoch ttheta_user_setpoint theta_user_setpoint noisy th_tth_permit\n",
" 4.5 9.0 9.998106479644775 10 9.0 4.5 0.0 31963.763745860513\n",
" 4.3500000000000005 8.700000000000001 10.893603086471558 11 8.7 4.35 0.0 34611.37532330056\n",
" 4.2 8.4 11.595271110534668 12 8.4 4.2 0.0 35180.93342784417\n",
" 4.05 8.1 12.299082040786743 12 8.1 4.05 0.0 37694.92329152548\n",
" 3.9 7.8 13.00736141204834 13 7.8 3.9 0.0 39442.76049515945\n",
" 3.75 7.5 13.71463131904602 14 7.5 3.75 0.0 42096.21324791565\n",
" 3.6 7.2 14.41884994506836 14 7.2 3.6 0.0 44406.552389513294\n",
" 3.45 6.9 15.12493348121643 15 6.9 3.45 0.0 45977.709130280506\n",
" 3.3000000000000003 6.6000000000000005 15.830729246139526 16 6.6 3.3 0.0 48881.20001936124\n",
" 3.15 6.3 16.53253173828125 17 6.300000000000001 3.1500000000000004 0.0 51082.06178377704\n",
" 3.0 6.0 17.24329447746277 17 6.0 3.0 0.0 53210.29908657747\n",
" #C Fri Jul 23 20:56:14 2021. num_events_baseline = 2\n",
" #C Fri Jul 23 20:56:14 2021. num_events_primary = 11\n",
" #C Fri Jul 23 20:56:14 2021. exit_status = success\n",
"\n",
" #S 345 scan(detectors=['noisy', 'th_tth_permit'], num=4, args='['theta', 3.5, 3.0, 'ttheta', 7, 6]', per_step='None')\n",
" #D Fri Jul 23 20:57:50 2021\n",
" #C Fri Jul 23 20:57:50 2021. plan_type = generator\n",
" #C Fri Jul 23 20:57:50 2021. uid = 5761047b-ac05-428a-8b9c-6601b0aba2c0\n",
" #MD uid = 5761047b-ac05-428a-8b9c-6601b0aba2c0\n",
" #MD beamline_id = Bluesky_training\n",
" #MD detectors = ['noisy', 'th_tth_permit']\n",
" #MD instrument_name = Bluesky Case Studies\n",
" #MD login_id = mintadmin@mint-vm\n",
" #MD motors = ('theta', 'ttheta')\n",
" #MD notebook = UB_autosave\n",
" #MD num_intervals = 3\n",
" #MD num_points = 4\n",
" #MD objective = Demonstrate UB matrix save & restore\n",
" #MD pid = 10389\n",
" #MD plan_pattern = inner_product\n",
" #MD plan_pattern_args = {'num': 4, 'args': [\"MyMotor(prefix='gp:m1', name='theta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint', 'backlash', 'backlash_velocity'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 3.5, 3.0, \"MyMotor(prefix='gp:m2', name='ttheta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint', 'backlash', 'backlash_velocity'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 7, 6]}\n",
" #MD plan_pattern_module = bluesky.plan_patterns\n",
" #MD proposal_id = training\n",
" #MD versions = {'apstools': '1.5.1', 'bluesky': '1.7.0', 'databroker': '1.2.3', 'epics': '3.5.0', 'h5py': '3.2.1', 'intake': '0.6.2', 'matplotlib': '3.3.4', 'numpy': '1.20.3', 'ophyd': '1.6.1', 'pyRestTable': '2020.0.3', 'spec2nexus': '2021.1.8'}\n",
" #P0 \n",
" #N 12\n",
" #L theta ttheta Epoch_float Epoch theta_user_setpoint theta_backlash theta_backlash_velocity ttheta_user_setpoint ttheta_backlash ttheta_backlash_velocity noisy th_tth_permit\n",
" 3.5 7.0 4.44653844833374 4 3.5 0.0 1.0 7.0 0.5 0.2 0.0 46002.45836582881\n",
" 3.333 6.67 8.672361612319946 9 3.3333333333333335 0.0 1.0 6.666666666666667 0.5 0.2 0.0 46875.28455439954\n",
" 3.1670000000000003 6.33 12.796637535095215 13 3.1666666666666665 0.0 1.0 6.333333333333333 0.5 0.2 0.0 50480.82899166115\n",
" 3.0 6.0 16.955684661865234 17 3.0 0.0 1.0 6.0 0.5 0.2 0.0 54033.337551435354\n",
" #C Fri Jul 23 20:58:08 2021. num_events_baseline = 2\n",
" #C Fri Jul 23 20:58:08 2021. num_events_primary = 4\n",
" #C Fri Jul 23 20:58:08 2021. exit_status = success\n",
"\n",
" #S 346 scan(detectors=['noisy', 'th_tth_permit'], num=4, args='['theta', 4.0, 3.0, 'ttheta', 8, 6]', per_step='None')\n",
" #D Fri Jul 23 20:58:28 2021\n",
" #C Fri Jul 23 20:58:28 2021. plan_type = generator\n",
" #C Fri Jul 23 20:58:28 2021. uid = 14e4398d-42b5-40cf-b082-e2f7e4fd07fd\n",
" #MD uid = 14e4398d-42b5-40cf-b082-e2f7e4fd07fd\n",
" #MD beamline_id = Bluesky_training\n",
" #MD detectors = ['noisy', 'th_tth_permit']\n",
" #MD instrument_name = Bluesky Case Studies\n",
" #MD login_id = mintadmin@mint-vm\n",
" #MD motors = ('theta', 'ttheta')\n",
" #MD notebook = UB_autosave\n",
" #MD num_intervals = 3\n",
" #MD num_points = 4\n",
" #MD objective = Demonstrate UB matrix save & restore\n",
" #MD pid = 10389\n",
" #MD plan_pattern = inner_product\n",
" #MD plan_pattern_args = {'num': 4, 'args': [\"MyMotor(prefix='gp:m1', name='theta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint', 'backlash', 'backlash_velocity'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 4.0, 3.0, \"MyMotor(prefix='gp:m2', name='ttheta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint', 'backlash', 'backlash_velocity'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 8, 6]}\n",
" #MD plan_pattern_module = bluesky.plan_patterns\n",
" #MD proposal_id = training\n",
" #MD versions = {'apstools': '1.5.1', 'bluesky': '1.7.0', 'databroker': '1.2.3', 'epics': '3.5.0', 'h5py': '3.2.1', 'intake': '0.6.2', 'matplotlib': '3.3.4', 'numpy': '1.20.3', 'ophyd': '1.6.1', 'pyRestTable': '2020.0.3', 'spec2nexus': '2021.1.8'}\n",
" #P0 \n",
" #N 12\n",
" #L theta ttheta Epoch_float Epoch theta_user_setpoint theta_backlash theta_backlash_velocity ttheta_user_setpoint ttheta_backlash ttheta_backlash_velocity noisy th_tth_permit\n",
" 4.0 8.0 5.639213562011719 6 4.0 0.0 1.0 8.0 0.5 0.2 0.0 37946.01826904763\n",
" 3.6670000000000003 7.33 10.168693780899048 10 3.6666666666666665 0.0 1.0 7.333333333333333 0.5 0.2 0.0 43240.06624621665\n",
" 3.333 6.67 14.600727081298828 15 3.3333333333333335 0.0 1.0 6.666666666666667 0.5 0.2 0.0 47339.97568055583\n",
" 3.0 6.0 19.046493530273438 19 3.0 0.0 1.0 6.0 0.5 0.2 0.0 52702.49931930254\n",
" #C Fri Jul 23 20:58:48 2021. num_events_baseline = 2\n",
" #C Fri Jul 23 20:58:48 2021. num_events_primary = 4\n",
" #C Fri Jul 23 20:58:48 2021. exit_status = success\n",
"\n",
" #S 347 scan(detectors=['noisy', 'th_tth_permit'], num=4, args='['theta', 4.0, 3.0, 'ttheta', 8, 6]', per_step='None')\n",
" #D Fri Jul 23 21:03:30 2021\n",
" #C Fri Jul 23 21:03:30 2021. plan_type = generator\n",
" #C Fri Jul 23 21:03:30 2021. uid = 8869043f-56fc-4b3b-ad75-bc024694e6d6\n",
" #MD uid = 8869043f-56fc-4b3b-ad75-bc024694e6d6\n",
" #MD beamline_id = Bluesky_training\n",
" #MD detectors = ['noisy', 'th_tth_permit']\n",
" #MD instrument_name = Bluesky Case Studies\n",
" #MD login_id = mintadmin@mint-vm\n",
" #MD motors = ('theta', 'ttheta')\n",
" #MD notebook = UB_autosave\n",
" #MD num_intervals = 3\n",
" #MD num_points = 4\n",
" #MD objective = Demonstrate UB matrix save & restore\n",
" #MD pid = 10389\n",
" #MD plan_pattern = inner_product\n",
" #MD plan_pattern_args = {'num': 4, 'args': [\"MyMotor(prefix='gp:m1', name='theta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint', 'backlash', 'backlash_velocity'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 4.0, 3.0, \"MyMotor(prefix='gp:m2', name='ttheta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint', 'backlash', 'backlash_velocity'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 8, 6]}\n",
" #MD plan_pattern_module = bluesky.plan_patterns\n",
" #MD proposal_id = training\n",
" #MD versions = {'apstools': '1.5.1', 'bluesky': '1.7.0', 'databroker': '1.2.3', 'epics': '3.5.0', 'h5py': '3.2.1', 'intake': '0.6.2', 'matplotlib': '3.3.4', 'numpy': '1.20.3', 'ophyd': '1.6.1', 'pyRestTable': '2020.0.3', 'spec2nexus': '2021.1.8'}\n",
" #P0 \n",
" #N 12\n",
" #L theta ttheta Epoch_float Epoch theta_user_setpoint theta_backlash theta_backlash_velocity ttheta_user_setpoint ttheta_backlash ttheta_backlash_velocity noisy th_tth_permit\n",
" 4.0 8.0 5.540862798690796 6 4.0 0.0 1.0 8.0 0.5 0.2 0.0 37996.6450939875\n",
" 3.6670000000000003 7.33 9.968387842178345 10 3.6666666666666665 0.0 1.0 7.333333333333333 0.5 0.2 0.0 42649.32391462264\n",
" 3.333 6.67 14.498717546463013 14 3.3333333333333335 0.0 1.0 6.666666666666667 0.5 0.2 0.0 46995.88551308093\n",
" 3.0 6.0 19.0245144367218 19 3.0 0.0 1.0 6.0 0.5 0.2 0.0 54150.34088276983\n",
" #C Fri Jul 23 21:03:49 2021. num_events_baseline = 2\n",
" #C Fri Jul 23 21:03:49 2021. num_events_primary = 4\n",
" #C Fri Jul 23 21:03:49 2021. exit_status = success\n",
"\n",
" #S 348 scan(detectors=['noisy', 'th_tth_permit'], num=4, args='['theta', 4.0, 3.0, 'ttheta', 8, 6]', per_step='None')\n",
" #D Fri Jul 23 21:04:04 2021\n",
" #C Fri Jul 23 21:04:04 2021. plan_type = generator\n",
" #C Fri Jul 23 21:04:04 2021. uid = e9498d10-8fb7-41c4-aaba-f5204db32761\n",
" #MD uid = e9498d10-8fb7-41c4-aaba-f5204db32761\n",
" #MD beamline_id = Bluesky_training\n",
" #MD detectors = ['noisy', 'th_tth_permit']\n",
" #MD instrument_name = Bluesky Case Studies\n",
" #MD login_id = mintadmin@mint-vm\n",
" #MD motors = ('theta', 'ttheta')\n",
" #MD notebook = UB_autosave\n",
" #MD num_intervals = 3\n",
" #MD num_points = 4\n",
" #MD objective = Demonstrate UB matrix save & restore\n",
" #MD pid = 10389\n",
" #MD plan_pattern = inner_product\n",
" #MD plan_pattern_args = {'num': 4, 'args': [\"MyMotor(prefix='gp:m1', name='theta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint', 'backlash', 'backlash_velocity'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 4.0, 3.0, \"MyMotor(prefix='gp:m2', name='ttheta', settle_time=0.0, timeout=None, read_attrs=['user_readback', 'user_setpoint', 'backlash', 'backlash_velocity'], configuration_attrs=['user_offset', 'user_offset_dir', 'velocity', 'acceleration', 'motor_egu'])\", 8, 6]}\n",
" #MD plan_pattern_module = bluesky.plan_patterns\n",
" #MD proposal_id = training\n",
" #MD versions = {'apstools': '1.5.1', 'bluesky': '1.7.0', 'databroker': '1.2.3', 'epics': '3.5.0', 'h5py': '3.2.1', 'intake': '0.6.2', 'matplotlib': '3.3.4', 'numpy': '1.20.3', 'ophyd': '1.6.1', 'pyRestTable': '2020.0.3', 'spec2nexus': '2021.1.8'}\n",
" #P0 \n",
" #N 12\n",
" #L theta ttheta Epoch_float Epoch theta_user_setpoint theta_backlash theta_backlash_velocity ttheta_user_setpoint ttheta_backlash ttheta_backlash_velocity noisy th_tth_permit\n",
" 4.0 8.0 5.37143349647522 5 4.0 0.0 1.0 8.0 0.5 0.2 0.0 38949.839143231766\n",
" 3.6670000000000003 7.33 9.859706163406372 10 3.6666666666666665 0.0 1.0 7.333333333333333 0.5 0.2 0.0 43461.79736667226\n",
" 3.333 6.67 14.294379949569702 14 3.3333333333333335 0.0 1.0 6.666666666666667 0.5 0.2 0.0 47545.930496638146\n",
" 3.0 6.0 18.838080167770386 19 3.0 0.0 1.0 6.0 0.5 0.2 0.0 53464.77967307891\n",
" #C Fri Jul 23 21:04:23 2021. num_events_baseline = 2\n",
" #C Fri Jul 23 21:04:23 2021. num_events_primary = 4\n",
" #C Fri Jul 23 21:04:23 2021. exit_status = success\n",
"\n",
" "
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "0566c766",
"metadata": {},
"source": [
"## User Code File\n",
"\n",
"A common request from instruments is to allow the user to write some python code that can be loaded into the current session. Since development of this *user code* is often iterative, it must be possible to reload the code without requiring the session to exit and restart.\n",
"\n",
"The local file `user/quick_hello.py` provides an example of such a user code file. Load this file with the command:\n",
"\n",
" %run -im user.quick_hello\n",
"\n",
"\n",
"\n",
"The `%run` is an [IPython Magic](https://ipython.readthedocs.io/en/stable/interactive/magics.html) command. _Magic_ commands are only available from the _interactive_ IPython command line (either in a console or Jupyter notebook). You can't use them in your Python files, functions, classes, etc.\n",
"\n",
"The `user/` directory must be on the import path, which we did as part of starting the `instrument` package (above). Also, `user/` _must_ contain a file named [__init__.py](./_about_init_files.md) so that `user` is an importable Python package.\n",
"\n",
"The `m` (as module) command option was added so it is not necessary to give the Python extension `.py`.\n",
"\n",
"Alternatively, we could use the command:\n",
"\n",
" %run -i user/quick_hello.py\n",
"\n",
"which loads a Python file from a directory.\n",
"\n",
"\n",
"SPEC users
\n",
"\n",
"For SPEC users, the `%run -i directory/file` syntax is the IPython (and Jupyter notebook) equivalent of SPEC's `qdo spec_macro.mac` command.\n",
"\n",
" \n",
"\n",
" \n",
"\n",
"This file demonstrates the quintessential [_Hello, World!_](https://en.wikipedia.org/wiki/%22Hello,_World!%22_program) demonstration in the context of the Bluesky framework. It defines an ophyd `HelloDevice` class which is used to create a Python object for control: `hello_device`. Finally, it creates a `hello_world()` (bluesky) plan which can be used with the bluesky RunEngine to run the _Hello, World!_ demonstration.\n",
"\n",
"First, load the code from this file:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "bd14bc01",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Loading 'Hello, World!' example.\n"
]
}
],
"source": [
"%run -im user.quick_hello"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "3f57c82c",
"metadata": {},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'findpv' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31mNameError\u001b[0m\u001b[0;31m:\u001b[0m name 'findpv' is not defined\n"
]
}
],
"source": [
"findpv(\"gp:userCalc8.CALC\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "068ea76b",
"metadata": {},
"source": [
"Observe that the same `.calculation` is both readable and writable from two different ophyd objects: `calcs.calc8` and `temperature`."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "0f7712ba",
"metadata": {},
"source": [
"## databroker\n",
"\n",
"The [databroker](https://blueskyproject.io/databroker) package provides \n",
"a Python interface to the database with the experiment data, including \n",
"references to the large file content such as area detector images. A \n",
"[YAML](https://yaml.org) configuration file connects databroker with a \n",
"specific repository in the MongoDB database server. In the example here \n",
"([bluesky_class.yml](./bluesky_class.yml)), the name of the *catalog* entry\n",
"in this file is `class_2021_03`. It makes two connections to a MongoDB server\n",
"running on the same workstation `localhost`. Both connections are to the \n",
"same MongoDB collection: `class_2021_03-bluesky`. The name of the file\n",
"is not important as long as it is placed in a directory searched by \n",
"`databroker.catalog`.\n",
"\n",
"**Example**\n",
"\n",
"```yaml\n",
"# file: bluesky_class.yml\n",
"# purpose: Configuration file to connect Bluesky databroker with MongoDB\n",
"# For 2021-03 Python Training at APS\n",
"\n",
"# Copy to: ~/.local/share/intake/bluesky_class.yml\n",
"# Create subdirectories as needed\n",
"\n",
"sources:\n",
" class_2021_03:\n",
" args:\n",
" asset_registry_db: mongodb://localhost:27017/class_2021_03-bluesky\n",
" metadatastore_db: mongodb://localhost:27017/class_2021_03-bluesky\n",
" driver: bluesky-mongo-normalized-catalog\n",
"```\n",
"\n",
"When the bluesky session starts, this `class_2021_03` catalog is referenced when\n",
"creating the `db` object in the instrument package, in \n",
"`instrument.framework.initialize.py` by these lines:\n",
"\n",
"```python\n",
"catalog_name = \"class_2021_03\"\n",
"db = databroker.catalog[catalog_name].v1\n",
"logger.info(f\"using databroker catalog '{catalog_name}'\")\n",
"```"
]
}
],
"metadata": {
"interpreter": {
"hash": "60aa360bcd8d3c8cfbc4e726e53a455fcd5c15cdf29caaf63c7ca2494eba79e9"
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
}
},
"nbformat": 4,
"nbformat_minor": 5
}