From pgf@acis Thu Sep 23 11:12:13 1999 Return-Path: Received: from acis.mit.edu (ACIS.MIT.EDU [18.75.0.187]) by wiwaxia.mit.edu (8.9.3/csr-2.4) with SMTP id LAA18595 for ; Thu, 23 Sep 1999 11:12:12 -0400 (EDT) Message-Id: <199909231512.LAA18595@wiwaxia.mit.edu> Received: by acis AA18059; Thu, 23 Sep 99 11:12:06 EDT Date: Thu, 23 Sep 99 11:12:06 EDT From: "Peter G. Ford" To: sot@head-cfa.harvard.edu, mm, dd Subject: Re: OTG Ops Summary meeting (ASCII) Cc: asc Status: R Content-Length: 18618 > From: Dan Muleady > To: SOT@head-cfa.harvard.edu, mm, dd > Subject: FW: OTG Ops Summary meeting > Date: Thu, 23 Sep 1999 06:33:21 -0700 > > A 1-hour telecon has been set up to summarize and discuss Chandra > OTG operations up to now. The discussion material is attached (word > document) > > The meeting is on Friday, Sept. 24 at 10 am EDT in the Action Room. > There is a meet-me line set up for those not on site at the OCC: > 1-888-394-8594 pass code 55758. > > Talk to you all later, > > Dan. > > ------------------------------------------- > Daniel J. Muleady > TRW Chandra X-Ray Observatory Operations > R9/2444D (310) 812-9951 > Pager: (310) 389-7760 > dan.muleady@trw.com ASCII version follows... ____________________________________________________________________________ OTG Operations Summary Chandra FOT/TST ____________________________________________________________________________ * OTG hardware commands worked flawlessly for both LETG and HETG - Rigorous hardware checked gave confidence to proceed with software operations * OTG Software is now working nominally for both LETG and HETG - OTG FSW successfully aborted each case where limit checks were violated - OTG FSW successfully completed insertions/retractions where limits were not violated and timing delay was adequate. * OTG Software Timing (Analysis by B. Twambly, B. Oliver and A. Roteliuk) - Time delays selected in RTS and Daily Load (6 minutes, 10.976 major frames) turned out to be borderline for success -- Major frame boundaries create uncertainty in OTG FSW timing - Software contains 24 minor frames (6.15 seconds) of latent delay prior to grating movement - Once grating movement is started, software waits 10 major frames before disabling -- check is GREATER THAN 10 major frames added to the major frame in which the motion started -- Enabling motion within 24 minor frames of a major frame boundary will result in motion starting in the next major frame, adding another major frame to total time required for FSW to process -- Time delay between enable and disable commands must be no less than 11 major frames + 24 minor frames + enough minor frames to complete processing - OTG.ATS and Ground SW RTSs revised to increase time delay to 9 minutes. ____________________________________________________________________________ Summary of OTG Hardware and Software Tests * Hardware Tests - Component acceptance tests (thermal , vibe, functional, etc) - 5000 cycle life test on fully assembled mechanism in thermal vacuum - Integrated system clearance tests before vibe test and again at final assembly on the HRMA - Integrated test with the OTG mechanism, MCE, EM RCTU, and EM CTU at Kodak -- This is where the OTG hardware interlocks and MCE were first jointly tested -- Flight software and simultaneous A and B side telemetry were not available - Integrated OTG and ACC mechanism vibe and modal test with OTG mass simulators - Breakaway torque tests of the final assembly on the back of the HRMA (x-axis vertical) - 1st motion testing of OTG mechanism drive arm after integration at TRW to verify commanding and limited mechanism motion and direction. * Software Tests - Every decision branch executed within Software Test Environment - Software Testing with MCE Hardware not possible in flight configuration * Software / Hardware Interfaces with MCE with both HETG and LETG are proven - Additional testing with the spacecraft not warranted or desired ____________________________________________________________________________ Summary of Key OTG Operations Events and Subsequent Corrections 1. 242:03:20 GMT, Failed First SW HETG Insertion - SCS 247 and 248 commands in incorrect order -- SCSs were changed to correct order and to increase time delay between grating move enable command and disable command to 6 minutes. - FSW dump of OTG data showed telemetry data being requested from wrong RCTU -- FSW Patch PR-76 corrected telemetry requests 2. 244:04:25 GMT Failed re-attempt of SW HETG Insertion - Traced to incorrect scale factor K-constant for OTG Motor Drive Voltage -- FSW Patch PR-78 corrected scale factor 3. 246:19:00 GMT, Failed re-attempt of SW HETG Insertion - MCE state did not get reset after previous abort -- Corrected MCE state by real-time commands. -- Successful re-insertion of HETG. 4. 255:03:45 GMT, Running P13 Load, LETG Failed to Insert - FSW aborted the move. - Traced to 253:08:15 GMT Running P12D load, LETG retracted, but MCE power was left ON - CAP 415 executed to insert LETG 5. 256:04:36 GMT, Running P13 load. LETG Retracted per daily load, but MCE 5 and 28 volt power supplies were left ON. - 256:07:40 GMT, CAP 416 executed to turn off MCE 28 & 5 volts power supplies. (CAP did not reset the MCE execute relay states.) - CM CAP 426 generated to increase time delay from 6 minutes to 9 minutes 6. 257:21:12 GMT, Running P14b Load. HETG failed to insert. - MCE relay states had not been reset after failed LETG retraction and execution of CAP 416. - 257:22:28 GMT, CAP 432 executed to insert HETG from ground using SW commands - CAP 416 revised to 416A to include reset of MCE execute relay states along with MCE power OFF commands 7. 260:21:32 GMT, Running P15 load. HETG successfully retracted. Software data dump shows all flags nominal. ____________________________________________________________________________ Detailed Time History of OTG Events GMT times are approximate, and based on SOC logs, SOT logs and Timelines, and indicate the start of the event +-----------+------------------------------------------------+------+------+ | TIME, GMT | EVENT | HETG | LETG | +-----------+------------------------------------------------+------+------+ | 240:05:09 | CAP 341, Hardware insertion of HETG | IN | OUT | | 241:20:46 | CAP 346 to patch OTG processing with new | IN | OUT | | | angular check limits | | | | 242:02:45 | CAP 341, Hardware retraction of HETG | OUT | OUT | |~242:03:00 | CAP 341, First HETG Software insertion | OUT | OUT | | | attempt, failed. Evaluation of commands in | | | | | SCSs 247 and 248 indicated they were in the | | | | | incorrect order. | | | | 242:03:20 | CAP 361 executed to insert HETG via hardware | IN | OUT | | | commands. | | | |~242:04:11 | CAP 361 executed to retract HETG via hardware | OUT | OUT | | | commands prior to loss of comm. | | | | 242:14:24 | CAP 362 executed to dump OTG data. Discovered | OUT | OUT | | | that telemetry being requested from incorrect | | | | | RCTU | | | | 242:??:?? | CM CAP 364 to change ground-command OTG | OUT | OUT | | | insert/retract RTSs to re-order commands and | | | | | increase time delay to 6 minutes (slots 247 | | | | | for insertions, and 248 for retractions) | | | | 243:20:43 | CAP 367 to load and dump SCS 247 (HETG SW | OUT | OUT | | | Insert) and SCS 248 (HETG SW Retract.) These | | | | | are the "new" 6-minute RTSs | | | | 243:20:58 | CAP 368, FSW Patch Upload (PR-76) to correct | OUT | OUT | | | telemetry request in OTG processing | | | | | (incorrectly requesting data from SIM RCTU | | | | | instead of TS RCTU) | | | | 244:04:25 | CAP 365, re-attempt to insert HETG using | OUT | OUT | | | software commands. CAP not completed. Traced | | | | | to a scale factor K-constant for OTG motor | | | | | drive voltage that is too low a value. OTG | | | | | software correctly aborts HETG move. | | | | 245:17:48 | CAP 374, FSW Patch Upload (PR-77, PR-78). | OUT | OUT | | | PR-77 is for ACA threshold limits, PR-78 is | | | | | for the OTG scale factor change for OTG motor | | | | | drive voltage. | | | | 246:19:00 | CAP 371 executed to attempt to insert HETG | OUT | OUT | | | using software commands. Initial insertion | | | | | unsuccessful. Failure traced to incorrect | | | | | starting state based (MCE Inhibit state) on | | | | | previously unsuccessful insertion attempt. | | | | 246:19:33 | Real-time commands corrected MCE state. | IN | OUT | | | Subsequently, HETG successfully inserted using | | | | | SW commands | | | | 247:03:30 | CAP 371 step 6 to retract HETG using software | OUT | OUT | | | commands. Retraction is successful. | | | | 249:00:19 | CAP 381. Release LETG. Insert LETG using HW | OUT | IN | | | Commands. Grating select relay for B-side set | | | | | to investigate back EMF phenomenon. | | | | 249:12:10 | CAP 381, retract LETG using HW commands. | OUT | OUT | | 249:12:24 | CAP 381. Successful SW insert of LETG | OUT | IN | | 249:12:40 | CAP 381. Successful SW retract of LETG | OUT | OUT | | 252:13:00 | Running P12D load, successful LETG insertion | OUT | IN | | | with HRC-S observing | | | | 253:08:15 | Running P12D load, LETG retraction (This is | OUT | OUT | | | where MCE power was left ON) | | | | 255:03:45 | Running P13 Load, This is the time at which | OUT | OUT | | | the LETG should have inserted, but the FSW | | | | | aborted the move because the MCE power was | | | | | already ON. | | | | 255:11:15 | Running P13 Load. Comm pass started | OUT | OUT | | | ~255:11:00. Reported to SOC that dispersed | | | | | pattern not seen on HRC-S. Determined the LETG | | | | | did not get inserted. | | | | 255:17:42 | CAP 362 executed to dump OTG data. The abort | OUT | OUT | | | routine in the FSW resets MCE "execute relay" | | | | | and the power supply converters. | | | | 255:17:49 | CAP 415 executed to insert LETG using SW | OUT | IN | | | commands. The RTS contains 6 minute delay | | | | 256:00:43 | CAP 416 written to turn off MCE power | OUT | IN | | | converters if the daily load insertion / | | | | | retraction fails to complete properly. | | | | | Original CAP 416 only contained 2 of the | | | | | required 4 cleanup items necessary to set up | | | | | the OTG for proper operations. CAP Later | | | | | revised as 416A to include reset of MCE | | | | | inhibits | | | | 256:04:36 | Running P13 load. LETG Retracted per daily | OUT | OUT | | | load, but MCE voltages were left ON. | | | | 256:07:40 | CAP 416 (not 416A) executed to turn off MCE | OUT | OUT | | | power 28 volts, 5 volts. It does not reset the | | | | | OTG inhibits. | | | | 257:??:?? | CM CAP 426 to change OTG.ATS to increase | OUT | OUT | | | timing between insertion execution and disable | | | | | commands from 6 minutes to 9 minutes | | | | 257:21:12 | Running P14b Load. HETG failed to insert. MCE | OUT | OUT | | | relay states had not been reset after failed | | | | | LETG retraction and execution of CAP 416. | | | | | However, software abort correctly reset | | | | | "execute relay" and the power supply | | | | | converters | | | | 257:22:28 | CAP 432 executed to insert HETG from ground | IN | OUT | | | using SW commands, as daily load failed to | | | | | perform insertion. | | | | 260:21:32 | Running P15 load. HETG successfully retracted. | OUT | OUT | | | Software data dump shows all flags nominal. | | | +-----------+------------------------------------------------+------+------+ Below is partial summary of the FSW checks prior to moving a grating. If any of the checks do not pass, the OTG move is aborted. The FSW abort routine shuts down the MCE 5V logic, the MCE 28V motor supply, and inhibits HETG and LETG grating movement (software flags). There are additional checks after the grating move is completed, but not shown here. ____________________________________________________________________________ OTG FSW Pre-Power On Checks Prior to OTG Move Execution: 1. Check that the motor drive converter inhibit relay is off 2. Check that the logic supply converter inhibit relay is off 3. Check that the HETG logic override relay is set to "limit switches active" 4. Check that the LETG logic override relay is set to "limit switches active" 5. Check that the HETG execute relay is set to "execute disabled" 6. Check that the LETG execute relay is set to "execute disabled" 7. Check if HETG = SELECT that the HETG rotation angle indicates retraction / insertion - If HETG is = INSERT, then HETG Potentiometer indicates retracted - If HETG is = RETRACT, then HETG Potentiometer indicates inserted - LETG Potentiometer indicates presently retracted 8. Check if LETG = SELECT that the LETG rotation angle indicates retraction / insertion - If LETG is = INSERT, then LETG Potentiometer indicates retracted - If LETG is = RETRACT, then LETG Potentiometer indicates inserted - HETG Potentiometer indicates presently retracted ____________________________________________________________________________ If all checks are passed, then 5V logic power is enabled. OTG FSW Post-Power On Checks Prior to OTG Move Execution: 1. Check that the logic supply converter inhibit relay is on 2. Check that the logic supply voltage is greater than 4.5 V 3. If HETG = SELECT, Check the HETG retracted limit switch 1A state - If HETG = INSERT, Limit Switch 1A data indicates RETRACTED - If HETG = RETRACT, Limit Switch 1A data indicates NOT RETRACTED 4. If HETG = SELECT, Check the HETG inserted limit switch 2A state - If HETG = INSERT, Limit Switch 2A data indicates NOT INSERTED - If HETG = RETRACT, Limit Switch 2A data indicates INSERTED 5. If LETG = SELECT, Check that the HETG limit switch 1A = RETRACTED 6. If LETG = SELECT, Check that the HETG limit switch 2A = NOT INSERTED 7. Check that the HETG enable logic status monitor indicates the motor drive is disabled 8. If HETG = SELECT, Check that the LETG limit switch 1A = RETRACTED 9. If HETG = SELECT, Check that the LETG limit switch 2A = NOT INSERTED 10. If LETG = SELECT, Check the LETG retracted limit switch 1A state - If LETG = INSERT, Limit Switch 1A data indicates RETRACTED - If LETG = RETRACT, Limit Switch 1A data indicates NOT RETRACTED 11. If LETG = SELECT, Check the LETG inserted limit switch 2A state - If LETG = INSERT, Limit Switch 2A data indicates NOT INSERTED - If LETG = RETRACT, Limit Switch 2A data indicates INSERTED 12. Check that the LETG enable logic status monitor indicates the motor drive is disabled. ____________________________________________________________________________ If all checks are passed, FSW issues HW commands to the MCE FSW next verifies the MCE logic states: 1. If HETG = SELECT, - Check that the OTG select relay status monitor indicates the HETG is selected - Check that the OTG select relay status monitor indicates the LETG is not selected - Check that the HETG direction relay status monitor indicates the HETG is to be inserted / retracted - Check that the HETG enable logic status monitor indicates the motor drive is enabled 2. If LETG = SELECT, - Check that the OTG select relay status monitor indicates the HETG is not selected - Check that the OTG select relay status monitor indicates the LETG is selected - Check that the LETG direction relay status monitor indicates the LETG is to be inserted / retracted - Check that the LETG enable logic status monitor indicates the motor drive is enabled ____________________________________________________________________________ If all checks are passed, FSW enables motor drive power FSW next verifies the Motor Drive Power states: 8. Check that the motor drive converter inhibit relay is on 9. Check that the motor drive voltage is greater than 27.5 V ____________________________________________________________________________ If all checks are passed, FSW enables the execute relay. FSW next verifies the Execute Commmand: 1. If HETG = SELECT, - Check that the HETG execute relay is set to "execute enabled" - Check that the HETG enable logic status monitor indicates the motor drive is enabled 2. If LETG = SELECT, - Check that the LETG execute relay is set to "execute enabled" - Check that the LETG enable logic status monitor indicates the motor drive is enabled ____________________________________________________________________________ Further software checks are performed after the grating movement has completed ____________________________________________________________________________