Aggregator
MDB2L27H suspect for Hall B's position oscillation
After recovering beam to Hall B, Hall B staff noted their positions were oscillating more than before. We can see the same movement in the 2E02 x-position and in the energy locks in the SL, both 2L20 with FFB on and 2L19 with it off. We will try to swap the card and see if that helps, but if not, we will run like this and continue looking for what is the cause.
2L18 Recovered
HV did not come back right away. I went out to check for a breaker trip but found nothing out of the ordinary. The Comms screen off of the zone Combo screen was showing an AUXP fault. Tried clearing faults and bringing HV back one more time. Success. Zone recovered without issue after that.
Downtime Incident CLOSED: 2L18 dropped HV
Downtime Incident CREATED: Swapping MDB2L27H
2L18 aux power supply fault
Downtime Incident CREATED: 2L18 dropped HV
Adjusted HallBCLAS12Orb lock parameters
Reduced fudge factor from 0.25 -> 0.1 and restored period from 3 -> 5 to try and get position oscillations under control. It's for sure coming from upstream, I'm just trying to prevent the lock from overcorrecting and making it worse. No difference so far; I'm going to keep poking around.
"Hall D back to 200nA on JD80-213 45/135"
"Hall D to BNR for orientation change"
Operator SIGN IN
Operator SIGN IN
smartRAT Cavity Instability Detection Daily Report
To view waveforms from a timestamp:
ssh opsrfdaqnl1 "smartRAT -m DAQ -d /data/fast_daq/<timestamp>"
To view waveforms of the top 25 from this report:
ssh opsrfdaqnl1 "smartRAT_view_daily_top25 2026-08-06-2026-08-13"
Data directory: /cs/prohome/apps/s/smartRAT2/pro/fileio/pca_data/output_cavityOutDet/calibrated_cavitylevel_centralize_tsfresh_minimal_gmespmes/whole/rcntnp/2026-08-06-2026-08-13/PCA/
Autolog save
CREW CHIEF SIGN IN
THOMAS JEFFERSON NATIONAL ACCELERATOR FACILITY
ACCELERATOR OPERATIONS CREW CHIEF SIGN IN FORM
Date : 08-13-26
Time : 23:21:20
Day : Thursday
Shift : Owl
Crew Chief : Howard
Program Deputy: Park
Team : D
Arm : Chaudhary
Designated SSO: Christian
LERF Operator:
Operators : Price, Smith
This stamp certifies that the above Crew Chief has read/reviewed/received the following:
Crew Chief Shift Log, 2026-08-13 SWING
Start of shift: Recovery of 2L11 in progress.
Crew Chief Timesheet Summary: 13 Aug 2026 SWING
LERF finished ~2130; taken to Controlled Access
iocnl22rffcc4 Unresponsive - IOC rebooted and zone restored
I was contacted by AESRF to take a look at iocnl22rffcc4. It had gone unresponsive. On the console, the IOC was continuously logging "Still waiting for mbuf cluster" messages along with NTP "Host is unreachable" errors. Network buffer exhaustion and loss of connectivity.
Worked out a reboot plan for the FCC IOC with the Crew Chief and AESRF. Note that when an FCC IOC loses connectivity, the zone is still protected by the FCC firmware and interlock chassis.
FSD Masking Tool: Successful System UN-Masking for: BeamLossMonitor
User: anthony
Areas: All
BeamLossMonitor Devices:
ILM0I07
ILM0L01A
ILM0L01B
ILM0L02
ILM2D00A
ILM2D00B
ILM0L04A
ILM0L04B
ILM0L05
ILM0L06A
ILM0L08
ILM0R01
ILM4D00
ILM0R03
ILM0R04
ILM0R06
ILM0R08
ILM0R09
ILM1L01
ILM1L02
ILM6R08
ILM3S00
ILM7S00
ILM9S00
ILM5S10
ILM5E03
ILM7S01
ILM9A01
ILM3E02
ILM5A01
ILM1A04
ILM1A06
ILM1A09
ILM5A10
ILM1A16
ILM1A19
ILM3R10
ILM9R07
ILM1R09
ILM2L03
ILM5R10
ILM5R05
ILM2L01
ILM2T09
ILM










