Add acquisition automation modes and fine-tune pipeline

[cfremling]

Summary

• Implement three acquisition automation modes (MANUAL, AUTOMATIC, FULL_AUTOMATIC) controlled via the new acqmode command, enabling configurable hands-off target acquisition
• Integrate ngps_acq fine-acquisition algorithm in slicecamd (source detection, centroiding, TCS offsets, closed-loop acquisition) via C callback API
• Publish ACAM guiding/acquiring state and preserve guiding during offsets
• Add seq_progress_gui (X11+ZMQ) for real-time observation phase display (SLEW/SOLVE/FINE/OFFSET/EXPOSE)
• Add configuration parameters for acquisition timeouts and fine-tune settings

Files changed (19)

New files (4): slicecamd/ngps_acq.c, common/ngps_acq_embed.h, utils/seq_progress_gui.cpp, common/message_keys.h, run/seq-progress

Modified files (14): sequencerd/sequence.cpp, slicecamd/slicecam_interface.cpp, sequencerd/sequence.h, sequencerd/sequencer_server.cpp, slicecamd/slicecam_interface.h, Config/sequencerd.cfg.in, Config/slicecamd.cfg.in, acamd/acam_interface.cpp, utils/CMakeLists.txt, slicecamd/CMakeLists.txt, common/sequencerd_commands.h, common/slicecamd_commands.h, sequencerd/sequencerd.cpp, slicecamd/slicecam_server.cpp

Test plan

• Build with cmake .. && make -j$(nproc) — verify all 19 files compile cleanly
• Verify acqmode command accepts MANUAL/AUTOMATIC/FULL_AUTOMATIC
• Test acquisition state machine transitions: SLEW → SOLVE → FINE → OFFSET → EXPOSE
• Verify seq_progress_gui launches and displays phase updates via ZMQ
• Test fine-acquisition loop with slicecam source detection and TCS offsets
• Confirm ACAM guiding state is preserved during offset_goal operations

🤖 Generated with Claude Code

[astronomerdave]

This is a single thread foreground process with one loop that does everything in turn on each pass through the loop:

1. process X-window events
2. reads the ZMQ message buffer
3. it forces EVERY daemon to republish a snapshot of its telemetry, regardless if it didn't change
4. sends a TCP/IP socket command to sequencerd, requesting its state (which was in the published message in step 3)
5. sends a TCP/IP socket command to sequencerd, requesting its wait state (which was in the published message in step 3)
6. sends a TCP/IP socket command to sequencerd, requesting its acquisition mode (which was in the published message in step 3)
7. sends a TCP/IP socket command to acamd, requesting is acquisition mode (which was in the published message in step 3)
8. redraws the GUI

This must takes tens to hundreds of milliseconds and with the built-in sleep could take on the order of half a second to get through the loop. Since it is single threaded and pulls out only one ZMQ message at a time, the ZMQ buffer could fill faster than this loop can read it, causing it to get behind. A daemon will publish whenever there is a change in status so there are more messages than the GUI is forcing to be generated.

One benefit of the PUB-SUB model is that subscribers need only listen and will be told if something changes. This is forcing every daemon in the instrument to re-publish, generating excess network traffic, on top of the four TCP/IP socket writes that request redundant information.

[cfremling]

Addressed all of these points in the latest commit:

1. ZMQ buffer drain — process_zmq() now drains up to 50 messages per loop iteration using a non-blocking check
   (has_message_nonblock()), so bursts from daemon publishes are consumed promptly before a single draw() call.
2. Removed all 4 redundant TCP commands — poll_sequencer() (state, wstate, acqmode) and poll_acam() (acquire) are
   deleted entirely. All this data is already in the ZMQ topics the GUI subscribes to: seq_seqstate, seq_waitstate,
   seq_progress (contains acqmode), and acamd (contains ACAM_GUIDING/ACAM_ACQUIRE_MODE).
3. Snapshot requests rate-limited — request_snapshot() is now rate-limited to once per 5s, and maybe_poll()
   only requests snapshots when data is actually stale (10s without updates, checked every 15s). Removed the aggressive
   1s snapshot request during USER wait states.
4. ZMQ fd in select() — Added the ZMQ socket fd alongside the X11 fd in select(), so the loop wakes immediately when
   messages arrive instead of waiting the full 200ms timeout.

One required addition: since acamd doesn't publish a snapshot when do_acquire() fails internally (e.g., guiding lost
due to bad seeing/weather), added iface.publish_snapshot() in dothread_framegrab right after the acquire(TARGET_NOP)
call on failure. This ensures guiding state changes are immediately visible to ZMQ subscribers without polling.

[prkrtg]

@cfremling why do we need this GUI? Per the design document, we would just have the main observing GUI change it's status as needed based on the sequencer's state

[cfremling]

the observer needs to see what is going on in a more fine grained way, mainly in case there is a failure at some step, so that the observer can know what to do next. It would be ideal if the main gui did it (but I needed something to look at when developing the algorithm). If we can have the same or very similar features and indicator progress somewhere on the main gui I am all for it.