spacetelescope/catkit2: v0.7.0

catkit2 v0.7.0 Release Summary

Core Infrastructure & C++ Updates

Message Broker & Data Communication

• Add a shared-memory message broker
• Enhanced Message Subscription Handling: Improved MessageBroker and MessageSubscription classes for more robust property and command message routing with trace ID support
• Message Metadata Management: Extended Message class with comprehensive metadata entry support for tracking and debugging inter-service communication
• Topic-based Message System: Refined implementation of topic-based publish-subscribe pattern in LocalMessageBroker with support for both NewestOnly and Sequential message modes
• Property & Command Protocol: Refined message-based protocol for property gets/sets and command execution using MessageBroker topics with error handling

DataStream Architecture & Service Handling

• Stream Parameter Updates: Enhanced DataStream with dynamic UpdateParameters() method allowing runtime reconfiguration of data type, dimensions, and buffer size
• Frame ID Management: Improved frame tracking with refined SubmitFrame() and RequestNewFrame() APIs for better synchronization across service boundaries
• Performance Optimizations: Reduced overhead in frame submission with improved buffer handling modes (BM_NEWEST_ONLY, BM_OLDEST_FIRST_OVERWRITE)
• Service State Synchronization: Strengthened synchronization between services through improved heartbeat and state monitoring threads
• Trace ID Support: Added comprehensive trace ID propagation for request-response correlation across all service communications

New Features & Capabilities

Hardware Support Expansion

• Hamamatsu Camera Service: Added support for Hamamatsu cameras to expand imaging capabilities
• Thorlabs Kinesis Motors: New service for controlling Thorlabs motorized stages via Kinesis SDK
• Thorlabs MCLS1: Added Linux compatibility

Documentation Enhancements

• Expanded Documentation: Comprehensive new service documentation pages covering multiple hardware interfaces
• Contribution Guidelines: Formalized contribution guidelines for community development
• Benchmark Suite: Expanded performance benchmarking tools including datastream_submit, datastream_latency, event_latency, and message_broker benchmarks for continuous performance validation

What's Changed

• Fix cube motor service doc file name in index.rst by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/209
• Reset properties while acquisition in Allied Vision camera service by @erinpougheon in https://github.com/spacetelescope/catkit2/pull/207
• Add informations about needing to create an environment variable in the Thorlabs Cubes service doc by @erinpougheon in https://github.com/spacetelescope/catkit2/pull/212
• Add or fix missing service docs by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/213
• Add stages models to the thorlabs kinesis motors service by @erinpougheon in https://github.com/spacetelescope/catkit2/pull/214
• Fix Thorlabs Cube motors size of serial number's list by @erinpougheon in https://github.com/spacetelescope/catkit2/pull/215
• Bugfix/piezo voltage limits datastream by @erinpougheon in https://github.com/spacetelescope/catkit2/pull/221
• Modify Allied Vision Camera service to make it work with the new VimbaX software by @erinpougheon in https://github.com/spacetelescope/catkit2/pull/222
• Service docs work (Steiger) by @steigersg in https://github.com/spacetelescope/catkit2/pull/224
• Add services for Ocean Optics spectrometers by @johanmazoyer in https://github.com/spacetelescope/catkit2/pull/172
• Fix newport picomotor failures by @raphaelpclt in https://github.com/spacetelescope/catkit2/pull/232
• Add note about VimbaX Python API installation by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/234
• Add a service for Thorlabs MCLS1 light source by @raphaelpclt in https://github.com/spacetelescope/catkit2/pull/235
• Fix testbed port numbers by @ehpor in https://github.com/spacetelescope/catkit2/pull/239
• Add performance tracing by @ehpor in https://github.com/spacetelescope/catkit2/pull/204
• Refactor the MCLS1 service by @ehpor in https://github.com/spacetelescope/catkit2/pull/243
• Add CPU affinity and process priority control by @ehpor in https://github.com/spacetelescope/catkit2/pull/245
• Bugfix/mcls1 sim by @steigersg in https://github.com/spacetelescope/catkit2/pull/242
• Benchmarks in C++ by @ehpor in https://github.com/spacetelescope/catkit2/pull/236
• Add trace intervals for several services by @ehpor in https://github.com/spacetelescope/catkit2/pull/244
• Add DM base class. by @ehpor in https://github.com/spacetelescope/catkit2/pull/156
• Feature/steiger service docs round2 by @steigersg in https://github.com/spacetelescope/catkit2/pull/252
• Feature/4d service by @lanemeier7 in https://github.com/spacetelescope/catkit2/pull/255
• Remove VimbaPython dependency from environment.yml by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/257
• Make DM map and command functions to outside of ServiceProxy. by @ehpor in https://github.com/spacetelescope/catkit2/pull/260
• Fix typo in id property access of Hamamatsu camera by @johanmazoyer in https://github.com/spacetelescope/catkit2/pull/261
• Use entry points for services and service proxies by @ehpor in https://github.com/spacetelescope/catkit2/pull/251
• makes thorlabs_tsp01 temperature sensor heads optional per configuration by @lanemeier7 in https://github.com/spacetelescope/catkit2/pull/271
• Adding instructions for Apple Silicon by @RemiSoummer in https://github.com/spacetelescope/catkit2/pull/274
• Variable size buffer management by @ehpor in https://github.com/spacetelescope/catkit2/pull/265
• Update versions for upload-pages-artifact and deploy-pages actions. by @ehpor in https://github.com/spacetelescope/catkit2/pull/277
• Refactor synchronization structure by @ehpor in https://github.com/spacetelescope/catkit2/pull/263
• Enable high-resolution dither mode on BMC DMs by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/282
• Add service_type input to BMC child class init by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/284
• Add trace intervals to new DM services from #156 by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/280
• Remove old BMC services by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/285
• Add PCIe device id choice to DM startup by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/287
• MNT: Replace ubuntu-20.04 with ubuntu-22.04 (try 2) by @pllim in https://github.com/spacetelescope/catkit2/pull/288
• Move to pyproject.toml installation. by @ehpor in https://github.com/spacetelescope/catkit2/pull/275
• Add trace interval 'processing frame' to Hamamatsu camera by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/283
• Add shareable base class by @ehpor in https://github.com/spacetelescope/catkit2/pull/289
• Added missing standard libraries (string, stdint e.t.c.) to .cpp and header files by @alexisyslau in https://github.com/spacetelescope/catkit2/pull/296
• Add line to installation.rst describing how to increase ulimit by @steigersg in https://github.com/spacetelescope/catkit2/pull/294
• Finish adding missing service docs by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/279
• Fix compiler warning message. by @ehpor in https://github.com/spacetelescope/catkit2/pull/299
• Doc cleanup by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/298
• Refactor of Shareable states using a streaming specification. by @ehpor in https://github.com/spacetelescope/catkit2/pull/297
• Fix formatting in installation instructions by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/300
• Thorlabs VCP port and 4D detector mask by @RemiSoummer in https://github.com/spacetelescope/catkit2/pull/303
• Fix libabseil linking errors. by @ehpor in https://github.com/spacetelescope/catkit2/pull/305
• Fixing argument error by @Corentin-Pa in https://github.com/spacetelescope/catkit2/pull/307
• Add a shared-memory message broker. by @ehpor in https://github.com/spacetelescope/catkit2/pull/272
• Add an mtop utility. by @ehpor in https://github.com/spacetelescope/catkit2/pull/313
• Add missing includes. by @ehpor in https://github.com/spacetelescope/catkit2/pull/315
• Fix power setting in simulated NKT and MCLS1 Services by @steigersg in https://github.com/spacetelescope/catkit2/pull/317
• Move lock upstream to the deformable mirror base class. by @ehpor in https://github.com/spacetelescope/catkit2/pull/320
• updating the VCP port by @RemiSoummer in https://github.com/spacetelescope/catkit2/pull/326
• Hack for BMMC DMs on HiCAT with PCIe device setting by @steigersg in https://github.com/spacetelescope/catkit2/pull/328
• Submit zero to percent data stream upon startup of Thorlabs CLD by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/330
• Be strict about shared memory lifetime by @ehpor in https://github.com/spacetelescope/catkit2/pull/308
• Add a benchmark for latency for events. by @ehpor in https://github.com/spacetelescope/catkit2/pull/310
• Add an event based on futex by @ehpor in https://github.com/spacetelescope/catkit2/pull/311
• updates to shared memory tests by @RemiSoummer in https://github.com/spacetelescope/catkit2/pull/332
• Cleanup callbacks and properties after a Server or Service stops. by @ehpor in https://github.com/spacetelescope/catkit2/pull/322
• Add a message broker inside a testbed by @ehpor in https://github.com/spacetelescope/catkit2/pull/323
• Recreate shared memory for message broker if it already exists. by @ehpor in https://github.com/spacetelescope/catkit2/pull/335
• MTop should not create shared memory objects, only connect to them. by @ehpor in https://github.com/spacetelescope/catkit2/pull/341
• C++ deformable mirror base class by @ehpor in https://github.com/spacetelescope/catkit2/pull/329
• Add ArrayView type for message broker. by @ehpor in https://github.com/spacetelescope/catkit2/pull/331
• Add dark data stream to power meter by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/343
• Fix for HybridPoolAllocator wrongly removing pools. by @ehpor in https://github.com/spacetelescope/catkit2/pull/340
• Add new motor stage model by @Corentin-Pa in https://github.com/spacetelescope/catkit2/pull/347
• Fix compilation issue on Ubuntu 24.04.2 LTS by @a-sevin in https://github.com/spacetelescope/catkit2/pull/348
• Fix memory lifetime for Shareable objects. by @ehpor in https://github.com/spacetelescope/catkit2/pull/346
• Enable Services to publish their CPU and memory usage. by @ehpor in https://github.com/spacetelescope/catkit2/pull/345
• Use interface for MessageBroker. by @ehpor in https://github.com/spacetelescope/catkit2/pull/342
• Add tests for process stats. by @ehpor in https://github.com/spacetelescope/catkit2/pull/350
• Add Phasics SID4 Service to Catkit2 by @lanemeier7 in https://github.com/spacetelescope/catkit2/pull/338
• Use textual instead of curses for mtop. by @ehpor in https://github.com/spacetelescope/catkit2/pull/353
• logging INFO CYAN instead of BLUE by @lanemeier7 in https://github.com/spacetelescope/catkit2/pull/357
• Add service handling info by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/358
• ZWO Cool Camera Support by @lanemeier7 in https://github.com/spacetelescope/catkit2/pull/355
• Publish messages regarding service state and heartbeats. by @ehpor in https://github.com/spacetelescope/catkit2/pull/339
• Use the MessageBroker to communicate Properties of Services. by @ehpor in https://github.com/spacetelescope/catkit2/pull/352
• Add power function mode to Thorlabs CLD laser diodes by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/361
• Update CLD docs by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/362
• Adapt simulated CLD service by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/363
• Hamamatsu camera temperature cooling by @Corentin-Pa in https://github.com/spacetelescope/catkit2/pull/360
• Error in parameter name Hamamatsu by @johanmazoyer in https://github.com/spacetelescope/catkit2/pull/366
• Reset properties while acquisition in Hamamatsu camera service by @erinpougheon in https://github.com/spacetelescope/catkit2/pull/211
• Add Thorlabs rotation stages to Kinesis motor controls by @Lukasdly in https://github.com/spacetelescope/catkit2/pull/367
• Hamamatsu temperature safety by @johanmazoyer in https://github.com/spacetelescope/catkit2/pull/364
• Setting polarizer units by @Lukasdly in https://github.com/spacetelescope/catkit2/pull/370
• Add optical switch service by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/371
• Add attribution instructions by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/375
• Separate NKT SuperK services for EVO and FIANIUM by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/373
• Add proxy for optical fiber switch by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/374
• Adapt simulated service of optical switch by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/377
• Add default input channel for simulated optical switch by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/380
• Treat pulse picker ratio in sim service exactly like in hw service by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/382
• Clean up docs by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/381
• Fix data stream population in optical fiber switch service by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/385
• Feature/cookiecutter by @steigersg in https://github.com/spacetelescope/catkit2/pull/389
• Add Watchdog Service for System Monitoring and Slack Notifications by @lanemeier7 in https://github.com/spacetelescope/catkit2/pull/391
• Add cross-platform SIGINT handling in testbed.py by @a-sevin in https://github.com/spacetelescope/catkit2/pull/388
• Updating the VCP port (12/25). Adding test for catching behavior. by @RemiSoummer in https://github.com/spacetelescope/catkit2/pull/397
• Remove hardcoded vcp_port names and instead pull values from the config by @steigersg in https://github.com/spacetelescope/catkit2/pull/398
• Physik Stage Controller Service by @lanemeier7 in https://github.com/spacetelescope/catkit2/pull/401
• Bugfix/new 4d software compatibility by @steigersg in https://github.com/spacetelescope/catkit2/pull/394
• Add debug print function for message broker. by @ehpor in https://github.com/spacetelescope/catkit2/pull/395
• Fixes Issue: #405 by @MichaelPhilbin in https://github.com/spacetelescope/catkit2/pull/406
• Refresh stale documentation. by @ehpor in https://github.com/spacetelescope/catkit2/pull/413
• Remove licence from cookiecutter template by @alexisyslau in https://github.com/spacetelescope/catkit2/pull/418
• Updates to cookiecutter sample testbed by @steigersg in https://github.com/spacetelescope/catkit2/pull/419
• Updated the docs/testbed_implementation.rst by @alexisyslau in https://github.com/spacetelescope/catkit2/pull/414
• Started the documentation for thorlabs_mcls1 by @alexisyslau in https://github.com/spacetelescope/catkit2/pull/412
• Extend service docs by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/417
• Move execution of commands and the service proxy obtaining the API to a message broker. by @ehpor in https://github.com/spacetelescope/catkit2/pull/392
• Bugfix/service run cleanup by @lanemeier7 in https://github.com/spacetelescope/catkit2/pull/404
• Added a DataStream option for calling move_to. Issue #402 by @MichaelPhilbin in https://github.com/spacetelescope/catkit2/pull/407
• update cookiecutter variable name by @steigersg in https://github.com/spacetelescope/catkit2/pull/432
• Doc clarifications by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/438
• Clean up error handling in Newport XPS service by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/440
• Make picomotor current position data streams 64-bit by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/439
• thorlabs_mcls1: add OS-based backend selection (Windows UART lib / Linux serial) by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/441
• Picomotor 64-bit followup by @ivalaginja in https://github.com/spacetelescope/catkit2/pull/443

New Contributors

• @erinpougheon made their first contribution in https://github.com/spacetelescope/catkit2/pull/207
• @steigersg made their first contribution in https://github.com/spacetelescope/catkit2/pull/224
• @johanmazoyer made their first contribution in https://github.com/spacetelescope/catkit2/pull/172
• @lanemeier7 made their first contribution in https://github.com/spacetelescope/catkit2/pull/255
• @pllim made their first contribution in https://github.com/spacetelescope/catkit2/pull/288
• @alexisyslau made their first contribution in https://github.com/spacetelescope/catkit2/pull/296
• @Corentin-Pa made their first contribution in https://github.com/spacetelescope/catkit2/pull/307
• @Lukasdly made their first contribution in https://github.com/spacetelescope/catkit2/pull/367
• @MichaelPhilbin made their first contribution in https://github.com/spacetelescope/catkit2/pull/406

Full Changelog: https://github.com/spacetelescope/catkit2/compare/v0.6.1...v0.7.0