Changes
SHOC
,/* Change Log */
= SHOC User Manual =
* '''Observation planning tools:''' There is now a SHOC [http://shoc.saao.ac.za/SHOCsnr.html signal-to-noise ratio, exposure time, and limiting magnitude calculator]. * NOTE: THIS PAGE REFERS TO THE LINUX- and WEB-BASED SOFTWARE (AS OF MARCH, 2015). For the original, manufacturer-based software and instrument commissioning details, please refer to the manual at http://shoc.saao.ac.za/Documents/ShocnHelpful.pdf.
*'' If you have found this webpage already loaded on the browser, please refresh to ensure that it is current!!''
== SHOC Call Out list ==
'''Instrument software issues:''' Carel van Gend **050 (0768181312)
'''Telescope software issues:''' standby **113 (0212015178) or 9118
'''Electronics Issues:''' standby **103 (0212015179) or 9117
'''Mechanical Issues:''' standby **104 (0212015180)
* Please report any problems on [https://faultreports.saao.ac.za/ the fault reports page].
== Introduction ==
The SHOC systems consist of a filterwheel unit (with one or two filterwheels), a GPS unit for providing accurate timing triggers, an Andor iXon camera, and a control PC. Currently there are 2 full SHOC systems (shocnawe & shocndisbelief), and 1 development system running a different camera (shocnhorror). As of March 2015, the control software is web-based. Users point their browser to the instrument location, and separate tabs appear to allow control of the camera, GPS, and filter wheel. SHOC can be mounted on the 1.9- and , 1-m and Lesedi telescopes. We have tried to make the usage telescope specific, but it is very useful to know which SHOC instrument is mounted on the telescope you are using. Details on instrument operation are below.
The [http://shoc.saao.ac.za SHOC website] may well have more extensive detail about this instrument.
===IMPORTANT NOTES===* '''IMPORTANT NOTES ABOUT THE NEW SOFTWARECONSIDER DECREASING DETECTOR TEMPERATURE IF TAKING LONG EXPOSURES''' Reports in 2016 indicated that the dark current is higher than expected (by a factor of ~ 100; more information to follow). This is most noticeable for long (>=10 minute) exposures. The camera is automatically cooled to -50C. It can be cooled down to -70 C; however, if the ambient temperature is high, the cooler will be stressed and cannot reach minimum temperatures. If you do adjust the temperature to -60, -65 or -70, please listen carefully so that if the temperature alarm triggers (a high-pitched, continuous beep) you can stop cooling and prevent hardware failure. March 2017* '''THE NEW SOFTWARE HAS DIFFERENT FILE NAMES AND HEADER KEYWORDS THAN PREVIOUS DATA.''' If you use this information in your analyses, you will have to update it. The filename format has been standardized to <Instrument>_YYYYMMDD.####.fits, where the instrument is sha (shocnawe), shd (shocndisbelief), or SHH to designate which SHOC box is being used. Please check the header of a new data file or the wiki [http://topswiki.saao.ac.za/index.php/SHOC#FITS_Header_Keywords FITS header keywords] (the "new" keywords are directly from the Andor iXon camera under Linux). March 2015* '''THE FILE SIZE IS NO LONGER LIMITED TO 2 GB. ''' This means that if you take large data cubes, the file size has ''no limit'' (this is because we are no longer on a Window-based, 32-bit, operating system). If you end up with a very large file, data analysis can be difficult. We are working on an efficient method to automatically split files into smaller sizes, but for now users need to be aware that requesting a large number of images in one file could return a very large file size (and if you mistakenly delete your file, you have lost alot of work!). It is recommended that multiple files be manually taken. ''If you have any questions or problems with the data, feel free to contact carel@ saao.ac.za or amanda@ saao.ac.za.''* '''If you use the GPS, please set the pulse width to 10 microsec.''' It is currently defaulting to 1 microsec, and this will be fixed in the next version of the software.March 2015
== Change Log ==
'''Most recent chances changes at the top''' * '''Installation of "New" SHOC on the 74-inch, February 2024<br>'''** Users should also familiarise themselves with these two wikis if they are using SHOC on the 74-inch after February 2024** [https://topswiki.saao.ac.za/index.php/74%22_/_1.9m_-_%22Instrument_Selector%22 "Instrument Selector" wiki].** [https://topswiki.saao.ac.za/index.php/%22New%22_SHOC "New" SHOC wiki]. * '''v1.4.4 Release date 5 December 2018<br>'''** Bug: Camera user interface works without any filter wheels** Feature: Display filter wheel names on the user interface * '''v1.4.3 Release date 25 July 2018<br>'''** Bug: Fixed GPS antenna reference "off" bug by assigning per-telescope locations** Feature: Added a button to set GPS start time 30 seconds into the future (uses web browser operating system's time so make sure that your time zone is SAST) * '''v1.4.2 Release date 3 January 2018<br>'''** Feature: Better error reporting** Feature: TCS info now available when mounted on the 40-inch * '''v1.4.0 Release date 23 November 2016<br>'''** Bug: Camera interface doesn't revert to "manual mode" when reloading the page while it's in scripting mode** Feature: Upgraded to JS9 v1.10 ([https://github.com/ericmandel/js9/releases/tag/v1.10 Release Notes])** Feature: Display and plot total counts for active regions on image display * '''v1.3.5 Release date 16 November 2016<br>'''** Feature: GPS driver now has access to a general-purpose SHOC config file** Feature: GPS antenna delay is now set from the config file * '''v1.3.4 Release date 19 October 2016<br>'''** Bug: Default image display orientation. The image shown on-screen in conventional mode should now be correctly oriented. * '''v1.3.3 Release date 17 August 2016<br>'''** Bug: EM amplifiers are not selectable** Bug: 74" GPS antenna reference off (CLI only)** Bug: Fixed bug where start button says 'Start' instead of 'Stop'** Feature: GPS static and dynamic mode options (CLI only)** Feature: Shorter exposure times for kinetic series = 1 * '''v1.3.2 Release date 4 May 2016<br>'''** Bug: Filterwheel sometimes doesn't initialise on the first try.** Bug: Plot 1 point per exposure on analysis tab.** Feature: Turn off GPS pulses at restart_services.** Feature: Configurable audio signal when exposures are complete. * '''v1.3.1 Release date 6 April 2016<br>'''** Bug: If the output amp is switched, x should be flipped to the opposite of what it was.** Bug: Ambient temperature keword is now ENVTEM. No longer clobbers CCD temp keyword (TEMP).** Bug: TCS related keywords were not always correctly updated.** Bug: The SHOC cameras were referred to by the usual instruments to which they were attached. Now use serial number.** Bug: Fix to allow ampersand characters in additional header fields.** Bug: Scripting should now be working as expected.** Feature: Use logrotate to prevent log files from getting too big * '''v.1.3.0 Release date 10 February 2016<br>'''** Bug: Camera freezes up, requiring restart_services** Bug: When acquisition is stopped before first trigger is received, counter is nevertheless incremented.** Bug: Controller doesn't know about GPS when running in external mode** Bug: TCS information not appearing in FITS headers** Bug: Temperature reported as "N/A" when reloading camera interface** Feature: Swap "move" and "initialize" buttons on filter interface** Feature: Add additional binning options for scripting
* '''v1.2.2 Release date: 14 October 2015<br/>'''
*** Exposure label on interface now reset when changing from external to internal mode
*** Exposure time now set to 0 in external mode
*** A header [http://topswiki.saao.ac.za/index.php/SHOC#FITS_Header_Keyword_Conversion keyword conversion script] from the old headers (prior to v1.1) to the new headers (v1.1) to be able to run old data through [http://marissashoc.saao.ac.za/SHOCpipelinePipeline/ Marissa's updated pipeline].
*** New data entry under the ''Camera'' -> ''Advanced'' -> ''Additional'' Header Keywords for astronomer to enter header keywords related to target/RA/Dec/additional information
*** specifically for the 74in telescope, addition of FITS header keywords from the TCS. NOT AVAILABLE IN THE 40in TELESCOPE
== Quick Start ==
# Connect to the web browser. '''Go to http://shoc40in.suth.saao.ac.za:5000 or http://shoc74in.suth.saao.ac.za:5000 or http://shoclesedi.suth.saao.ac.za:5000 depending on which telescope you are using.'''# If you are prompted to log in, use the shoc74in or shoc40in accounts or shoclesedi username and password, depending on which telescope you are using.(passwords are the same as those used in the data transfer section, and should be obtained during startoff)
# Initialize the filter wheels and select your filter. '''You must initialize both filter wheels whenever the webservice is started.'''
# Click on camera tab: turn on the camera and confirm that it is cooling.
# Click on camera tab: set parameters on the camera and start previewing and taking data.
# Transfer data from the instrument to your local machine or the SAN.
# Make sure to turn OFF GPS trigger pulses when you are done with them.
# Turn OFF the camera before you leave!!
==Filter wheel==
* <span style="color: red">'''(1)'''</span> Point your browser at shoc40in.suth.saao.ac.za:5000 or shoc74in.suth.saao.ac.za:5000 or shoclesedi.suth.saao.ac.za:5000
'''NOTE: ON DNS ALIASES''': These telescope web addresses are aliases to the SHOC instruments. The real addresses for the instruments are shocnawe.suth.saao.ac.za:5000 or shocndisbelief.suth.saao.ac.za:5000. In order to avoid confusion, or the requirement that the user know which instrument is mounted on their telescope, we have created the aliases.
[[File:filterwheelFilterwheel.png|1000px|left]]<br clear=all>
* <span style="color: red">'''(2)'''</span>Select the filterwheel tab
* If using external triggering, select the GPS tab <span style="color: red">'''(1)'''</span>
* Ensure timing is valid <span style="color: red">'''(2)'''</span>.
* Set the Programmed Output Pulse (POP) parameters (mode <span style="color: red">'''(3)'''</span>, pulse width <span style="color: red">'''(4)'''</span>, start date <span style="color: red">'''(5)'''</span>, start time <span style="color: red">'''(6)'''</span>(<strong>NOTE</strong>: the "now + 30 seconds" button uses your browser's time so make sure your operating system is set to local time), trigger interval <span style="color: red">'''(7)'''</span>) and click apply <span style="color: red">'''(8)'''</span>. The status should change to pending <span style="color: red">'''(9)'''</span>.* Note that the current The recommended default pulse width is 1 microsec and it should be 10 microseconds. Please change it!
==Camera==
* Stop to abort <span style="color: red">'''(1)'''</span>, or wait until completion. The image display area (based on DS9) <span style="color: red">'''(2)'''</span> shows snapshots of the images throughout the acquisition.
* When the series is complete, the FITS data cube is copied to /data/<telescope>/<INSTR>/<YYYY>/<MMDD>/<filename>.fits, where:<br/>
:: '''<telescope>''' is 40in or 74inor lesedi,
:: '''<INSTR>''' could be shd for shocndisbelief, sha for shocnawe or shh for shocnhorror,
:: '''<YYYY>''' is the year,
When running a series of exposures on the same target with different filters it
can be useful to make use of the automated acquisition mode on the camera tab.'''NOTE: SCRIPTING ALWAYS USES THE INTERNAL TRIGGER'''
[[File:scripting_1.png|1000px]]
When you click the "New script" option a dialog will appear giving you
<span style="color: red;">'''(1)'''</span> a text field to enter a name for your new script. You can then enter any
name and click the <span style="color: red;">'''(2)'''</span> "Create Script" button to create a new one. ''NOTE: Please ensure that your script has an easily identifiable name for the observer and run, e.g. Gulbis_20150617Script1Sickafoose_20150617Script1. All scripts are currently being stored in the drop-down web interface -- this means you will see other people's scripts and will have to look there for your own in order to load the script in subsequent observing runs.''
Unless you click the <span style="color: red;">'''(1)'''</span> "Stop" button while the acquisition sequence is in
progress, the sequence will run to completion.
[[File:scripting_8.png|1000px]]<br>
Scripts can also be repeated by clicking the repeat button, entering the number of times the script should be repeated and clicking the <span style="color: red;">'''(2)'''</span> "Start" button.
==Data Transfer==
There is an automatic transfer of data from the instrument to the file servers in Sutherland and Cape Town, every morning at 7:30. So you should not need to worry about the security of your data. However, you'll still need to access your data! We describe here how to do this. There are two ways to transfer get the data: (1) directly to your computer from the instrument PC or (2) onto from the SAAO storage area network (SAN). The data are automatically copied to the server every morning (at 9:30am).* To use method (1) To transfer data directly to your computer, use the command line.:
** Open a shell on your local PC (Putty if you're using Windows or a bash/tcsh/sh if on Mac or Linux)
** Copy the data using the following command, depending on the telescope:
rsync -avzP shoc74in@shoc74in.suth.saao.ac.za:/data/74in/sh?/<YYYY>/<MMDD>/*.fits /LocalFilePath
rsync -avzP shoc40in@shoc40in.suth.saao.ac.za:/data/40in/sh?/<YYYY>/<MMDD>/*.fits /LocalFilePath
---- where sh? is a wildcard that will point to the correct instrument mounted on the telescope for that night and "/LocalFilePath" is the location on your local machine
---- the password is that for the shoc74in or shoc40in accounts
rsync -avzP ccd74@astro.suth.saao.ac.za:/data/telescopedata/74in/sh?/<YYYY>/<MMDD>/*.fits /LocalFilePath
rsync -avzP ccd40@astro.suth.saao.ac.za:/data/telescopedata/40in/sh?/<YYYY>/<MMDD>/*.fits /LocalFilePath
=== Web interface===
* The primary means of controlling SHOC is via a web interface. The Firefox or Chrome browsers are preferred.
* Point your browser at http://shoc74in.suth.saao.ac.za:5000 or http://shoc40in.suth.saao.ac.za:5000 or http://shoclesedi.suth.saao.ac.za:5000
* If you are prompted to log in, use the shoc74in or shoc40in accounts depending on which telescope you are using.
* The interface should appear, showing three tabs: Filterwheel, GPS and Camera.
* To set the Programmed Output Pulse, choose whether you want a single or repeating pulse, then choose a pulse width, the start date and time (in local time, 24 hour format), and for repeating pulses, the repeat interval. Then push the "Apply" button. Note: the POP may only be set to occur in the future; if an error occurs, check that you haven't chosen a time in the past.
* To stop the pulses from being sent, click the "Stop" button.
''* Note about timing accuracy: Running restart_services on the SHOC machine syncs the computer clock *and* displays the timing offset. The clock syncs automatically every evening, but if you are wondering about the computer clock accuracy you can run this script.''
===Camera===
* Below this, the readout speed and preamp gain may be set.
* Below this, the readout speed and preamp gain may be set.
* The last setting is the Electron Multiplying (EM) gain. <span style="color: red">NOTE: DO NOT USE ELECTRON MULTIPLYING MODE UNLESS YOU HAVE SPECIFICALLY OBTAINED PERMISSION FROM DR AMANDA GULBIS SICKAFOOSE TO DO SO, AND YOU UNDERSTAND HOW AND WHEN TO USE IT. PERMANENT DAMAGE TO THE SENSOR CAN RESULT FROM INCORRECT USAGE. If you are using EM mode, choose the EM output amplifier, and then select a gain. BEGIN WITH LOW GAIN VALUES, BECAUSE IF THERE IS TOO MUCH SIGNAL''' YOU CAN DAMAGE THE CAMERA.'''</span>
* When setting the horizontal shift speed, preamp gain and (if using EM mode, the EM gain), note the following: Higher readout speeds allow for shorter exposure times, but at the expense of noisier data, while higher electron to ADU gain values decrease the dynamic range. Read noise and electron to ADU gain values for each mode are provided in the PDF manual. In addition to this, it should be noted that as it takes about 6 milliseconds for the top-most pixel in the frame to be shifted to the storage area and SHOC does not have a shutter. The top row of pixels in each frame will have an exposure time of about six milliseconds longer than the bottom row. Consequently although it is possible (with enough subframing and binning) to expose for less than a hundredth of a second, this is inadvisable for the above reason. Finally, as indicated above, the 1MHz CON and EM modes are both 16bit. What this means is that the mode will reach saturation at 2^16 = 65536 counts. On the other hand, the 3 MHz CON and EM and 5,10 MHz EM modes are all 14-bit modes and saturate at 2^14 = 16384 counts. As the saturation limits of each mode will also depend on the electron to ADU gain, tables in the printed manual give the saturation limit in both counts and electrons for each mode.
* To get a quick view of the images that will result, press the "Preview" button. Images will be displayed in the left-hand display panel. Press "Stop" when you're done.
(vi) The regions will continue to appear on the display under the "Controls" GUI tab. Selecting "Region" -> "Delete Regions" will remove all of the regions shown on the frame.
[[File:RealTimeSHOC.jpg|800px|left]]<br clear=all>
==== Totcounts and bscounts ====
Totcounts are the total counts in the box region, while bscounts has the calculated background (see below) subtracted from the total.
==== Background and Noise ====
The background and noise are obtained from the outer 4 pixel-wide perimeter of the selected region. The background value is the median value of this area and the noise value is the RMS.
==== Centroid ====
The centroid is obtained by locating the brightest 9 pixel box and computing the centroid of the background subtracted data about this point.
==== FWHM ====
The FWHM is estimated from the image moments about the centroid, assuming a Gaussian distribution.
===FITS Header Keywords===
|EXPOSURE= 1.10916 / Total Exposure Time
|-
|TEMP = -999 23. / Temperature |The CCD temperature. If camera temp the temperature is not yet stabilized, this will be -999. and Then UNSTTMP gives the unstabilized CCD temperature will be under keyword "UNSTTEMP".
|-
|READTIME= 1.0E-06 / Pixel readout time
|UNSTTEMP= -35.764 / Unstabilized Temperature
|If camera temp is stabilized, this will be -999
and the temperature will be under keyword "TEMP".
|-
|A keyword that we are still trying to decipher.
When this camera setting is "off" the value of 116 is listed in the
header. When the setting is "on" the value is 0.
|-
|NSETHSLD= 0 / Number of Photon Counting Thresholds
|-
|FRMCNT = 1 / Frame Count
|-
|PORTMODE= 0 / Port Readout Mode
|-
|LSHEIGHT= 0 / Exposure Window Height
|-
|LSSPEED = 0. / Line Scan Speed
|-
|LSALTDIR= 0 / Alternating Readout Direction
|-
|LSCTRL = 0 / Scan Speed Control
|-
|LSDIR = 0 / Readout Direction
|-
|FKSMODE = 0 / Fast Kinetics Storage Mode
|-
|FKTMODE = 0 / Fast Kinetics Time Scan Mode
|-
|USERTXT1= ' ' / User text
|-
|ESHTMODE= 0 / Electronic Shuttering Mode
| |-|HIERARCH PREAMPGAINTEXT = ' ' / Pre-Amplifier Gain|-|HIERARCH SPECTROGRAPHSERIAL = ' ' / Spectrograph Serial |-|HIERARCH SHAMROCKISACTIVE = 0 / Shamrock is active |-|HIERARCH SPECTROGRAPHNAME = ' ' / Spectrograph name |-|HIERARCH SPECTROGRAPHISACTIVE = 0 / Spectrograph is active |-|HIERARCH IRIGDATAAVAILABLE = 0 / IRIG Availability |-|IRIGDATA= ' ' / IRIG Timestamp| This is the last keyword of those coming straight from the Andor Ixon camera.
|-
|AIRMASS = 'NA ' / The airmass (sec(z))
|DOMEPOS = 'NA ' / The dome position
| Read from the TCS.
|-
|ENVTEM = '26.65 ' / The average ambient temperature
| This is the temperature on the plateau
|-
|FILTERA = ' ' / The active filter in wheel A
| This should be a human-readable name for the filterin wheel A
|-
|FILTERB = ' ' / The active filter in wheel B
| This should be a human-readable name for the filterin wheel B
|-
|GPS-INT = ' ' / GPS trigger interval (msec)
|-
|POSA = '7 ' / The slot position of wheel A
| The numerical slot position of wheel A
|-
|POSB = '8 ' / The slot position of wheel B
| The numerical slot position of wheel B
|-
|TELDEC = 'NA ' / The telescope declination
|ZD = 'NA ' / The telescope zenith distance
| Read from the TCS. Currently only on the 74in!
|-
|AIRMASS = 'NA ' / The airmass (sec(z))
|-
|HUMIDIT = '9.86 ' / The average humidity
|-
|RELSKYT = '49.57 ' / The relative sky temperature (average cloud cov
|-
|SEEING = 'Unknown ' / The curent seeing
|-
|TMTDEW = '34.44 ' / Difference between current and dew-point temper
|-
|WIND = '39.73 ' / The average wind speed
|-
|END
'''NOTE:''' The conversion script makes use of [http://astropy.org Astropy]. See the official documentation for [http://astropy.readthedocs.org/en/stable/install.html installation instructions].
===LED timing tests===
''This functionality should be available on the 74", the 40" and Lesedi. It has only been tested on the 74".''
The idea is to take a long, GPS-triggered data cube with SHOC at high cadence, while running the LED at 1 PPS (which has a longer pulse on the minute and can thus be used to test absolute timing accuracy).
*The dome and mirror covers must be OPEN for this test to work. (The GPS needs to get satellite signal, and the LED must be seen through the light path.)
*There is an LED mounted in the telescope optical path that can be used to independently verify SHOC timing. Log into the following website:
http://10.2.55.5:4000/timepulse
Ask the Electronics standby for username and password. This website allows selection of LED pulses of different cadence (1, 2, 5, 10, 100 Hz and 100kHz), see the image below. '''Note that the pulse must be switched off when you are done, otherwise it will continue blasting photons into the telescope!'''
[[image:74inLEDPulseWebsite.jpg|500px]]
* Check that the LED is visible by turning on a pulse and seeing if it is apparent in the preview images. For example, take 0.2-s exposure and set the LED to 1 PPS, then look for increased counts/flashes on the display every 5th frame.
* '''To take a timing data cube:'''
** start the LED at 1 PPS
** in preview mode, increase the SHOC binning and readout speed to reach the desired exposure cadence (< 0.05s).
** set SHOC to external timing, select >=10000 frames, and start the GPS with repeated pulses at the chosen cadence
** REMEMBER TO TURN OFF THE LED PULSES WHEN DONE.
* '''Analysis:''' you are free to analyze your own timing data! Otherwise, an analysis pipeline has been written by Marissa Kotze and is supported by Carel van Gend. Carel can assist with data analysis.
* Download the SHOCpipeline.tar and PULSEtiming.tar software. Create a directory for the software (e.g. timingtests) and extract the software to that directory:
mkdir timingtests
cd timingtests
tar -xvf ../SHOCpipeline.tar
tar -xvf ../PULSEtiming.tar
* In the README file, follow the instructions on setting up iraf.
* Then make a further subdirectory for the testdata:
mkdir testdata
* Copy the data cube to this directory, and run the testPULSEtiming.py script:
cd testdata
python testPULSEtiming.py <SHOC fits file name>
* The software will examine the header, ask the user for confirmation of a few values, and then verify the timing. A report is printed out at the end.
== In-depth Information ==
== User's troubleshooting guide==
===Camera Issues===
* '''Camera won't switch on:''' If the camera display times out when trying to switch on, you will need to restart the services. To do this:
** Open a terminal. This is available from the top menu or side menu of the thin client in the warm room. If the web browser is full screen, push 'F11', and you should then see the top menu bar. Once the terminal window appears, ssh to the instrument by running the command
or
ssh shoc40in@shoc40in.suth.saao.ac.za
or ssh shoclesedi@shoclesedi.suth.saao.ac.za When prompted for the password, use the password for the 74in or 40in or lesedi user.
** In the terminal, type
restart_services
** Return to the web browser, and reload the page, then try to start the camera again.
** It may be necessary to restart the services a few times before you can start the camera.
* '''Acquisition doesn't start:''' Check on the GPS tab that the trigger signal has been activated. If it has, but the camera is still not taking images, try stopping the camera, resetting the GPS trigger to a time in the near future, and try again. Sometimes, it appears that the GPS trigger signal is not being sent (despite the software reporting that it is). The current acquisition should be stopped and restarted (because when the acquisition is started, the time the POP signal will start is recorded and will apear appear in the fits headers - so stopping and starting the acquisition will ensure that the correct POP time is used). Note that the index willl be incremented when doing this. To avoid this, you can rewind the increment by 1, in the advanced tab. Make sure you have no data that will be overwritten. *''' Acquisition doesn't finish:''' Sometimes it appears that the camera does not ever leave the acquiring state (despite all images in the kinetic series having been obtained). It may in this case be neccessary necessary to restart the server, manually edit the FITS files to include the filter and timing informationsinformation, and to then copy them to their final destination. The server may be restarted running the command "restart_services" from a terminal. The data is spooled to a directory under /data/spool/, based on the date, the sequence number and the time the acquisition started. In this directory is the data file, data.fits, and file containing other acquisition parameters, info.txt. The latter may be used to update the FITS header. The procedure is as follows:
** Rescue your data using the data rescue utility, as described in the section Rescuing Data.
** Run the command to restart the webserver, as described above in the subsection "Camera won't switch on".
*'''Camera overheats:''' Should the camera over heat (this normally results from the air outlets being blocked), an alarm will sound from the camera. Unfortunately no indication or reason for the alarm will be given on the control software. In the case of this happening please shut the instrument down and contact electrical support immediately. THE CAMERA MAY NOT BE USED WHILE THE ALARM IS SOUNDING.
===GPS Issues===
* '''The GPS time is not valid:''' If the GPS reports that the timing is not valid, check that it is able to see at least a few satellites, and that the signal quality is acceptable. Click on the Receiver Status button on the GPS tab, and a pop-up should appear, displaying the signal quality of the visible satellites. If each signal quality bar is zero, check that the dome is open; the antenna can't detect the satellites if it's closed. If there are a number of satellites visible, each with a reasonable signal, you might need to wait for a while for the GPS to rgister this and for the time to become valid. Normally this happens quite quickly, but for reasons we don't entirely understand, sometimes this can take five or ten minutes.
It may also help to put the GPS into dynamic mode. Usually the GPS is set to operate in static mode, where the known position of the telescope is used. In dynamic mode, the GPS will first locate itself before getting a lock on the time. It may be that doing this will help to get a lock on the time.
To put the GPS into dynamic mode, or to return it to static mode, use the command line interface described below. (One can also return the GPS to static mode by doing a "restart_services" call.
To use the command line interface:
** Fire up a terminal
** $ ssh <user>@<shocndisbelief|shocnawe>.suth.saao.ac.za
** $ shoc-gps-cli
** The commands in the cli are help, status, pop, mask, bias, timing_mode and location. Use "help <command>" for more information.
** The timing_mode option makes the GPS use static or dynamic mode.
** The status option prints out all the current status reported by the GPS. If the command line tool reports an error while updating the status, simply try again until it works.
** type "q" to exit the interface
=== Dealing with large FITS files===
When taking very large data cubes, the file sizes may grow unmanageably large. On older 32-bit systems, FITS files were automatically truncated at 2GB and rolled over, but on the new 64-bit systems we use, this does not happen. Consequently, very large FITS files may be generated. To cope with these, there is a utility called split_fits.py. This script prompts for the name of a FITS file to split, and the number of chunks into which it should be split. It then splits the large file into this number of smaller files. The FITS header is identical in each of the resulting files.
The script split_fits.py is installed on both astro.cape and astro.suth, in the directory /usr/local/saao/bin. To run it, type
split_fits.py
then when prompted enter the filename and number of chunks to produce. The output files are put in the same directory as the input file.
=== Rescuing Data===
N. B. It is NOT normally necessary to run the data rescue script described below; the problem we had with the interface freezing up and acquisitions not being completed is now solved.
Under normal operation, the system automatically updates the FITS cube header with supplementary information (filter, external trigger parameters, telescope pointing information etc.) and the moves the cube to the permanent (for the night) storage area. From time to time, it may be necessary to do this manually, using the rescue_data.py tool. In particular, if the system has frozen at the end of an acquisition, but before fixing the header and moving the data file, it will be necessary to perform this procedure. ''This procedure does NOT need to be done when stopping an acquisition; only if the interface hangs and you cannot do anything do you need to run this script.''
or
ssh shoc40in@shoc74in.suth.saao.ac.za
or
ssh shoclesedi@shoclesedi.suth.saao.ac.za
When prompted for the password, use the password for the 74in or 40in user. Then go to the directory containing the spooled files:
cd /data/spool
sudo rescue_data.py
and then answer the questions as prompted. You'll be prompted for a password, use the password for the user you logged in as.
* The telescope is the name of the telescope being used; either 40in or , 74inor lesedi.
* The instrument will be one of shocnawe (sha) or shocndisbelief (shd). These are used to create the directory and filename in the /data directory. The directory will be of the form /data/<telescope>/<lowercase_instrument>/<YYYY>/<MMDD>, and the file of the form <instrument>_<YYYYMMDD>.<index>.fits. From this, the instrument name can be determined. At the moment, SHA is the SHOC instrument mounted on the 74inch and SHD is on the 40in.
* You will be prompted for an index to use. NB: '''make sure there is no existing file with the same index that will be overwritten''' The script will scan the target directory and list the existing files. Choose and index which isn't used(one option is to use an index like 3a , if index 3 exists). In other words, if files such as SHA_20150403.0004.fits and SHA_20150403.0005.fits, choosing either index 4 or 5 will result in that file being overwritten. If you choose as index 4a or 5a, the files will not be overwritten.
= Technician's guide =
'''IMPORTANT: If a SHOC machine is swapped out and installed on a different telescope, the website aliases (shoc74in and shoc40in) and data storage paths will be *incorrect*. Please notify the IT standby person before trying to use them. Currently, shocnawe is tied to the 74in and shocndisbelief is tied to the 40in.'''
==Mounting SHOC ==
*The SHOC box should be mounted at the base of the telescope, and the camera should be mounted below the filter wheel box.
*** Use the up and down arrows to view each line of the file. Change the telescope name to the name of the currently used telescope: 74in or 40in. The instrument name should already be set to sha or shd (for shocnawe or shocndisbelief) and the log level should be info (not debug). In the editor, to remove a character type x . To add a character type a followed by the character to add, then hit esc to stop inserting. To exit, type :wq .
* To enact any changes made during configuration, restart all services:
**Under /home/ccd/programming/shoc/bin/, type the following: ./restart_services ==Switching the SHOC computer to a new telescope== Notify the IT standby person, so that the DNS records can be altered - they should use the information below. '''Normal Setup''' {| class="wikitable" border="1"|-! Name! Type! Content! TTL|-| shocnawe.suth.saao.ac.za| A| 10.2.2.89| 86400|-| shocndisbelief.suth.saao.ac.za| A| 10.2.2.88| 86400|-| shoc40in.suth.saao.ac.za| CNAME| shocndisbelief.suth.saao.ac.za| 360|-| shoc74in.suth.saao.ac.za| CNAME| shocnawe.suth.saao.ac.za| 360|-| shoclesedi.suth.saao.ac.za| CNAME| shocndisbelief.suth.saao.ac.za| 360|} The shoc computer should be reachable by an '''alias''' containing the telescope name. If, e.g. '''shocndisbelief''' is mounted on the '''74inch''' telescope, the alias '''shoc74in''' should be changed to point to '''shocndisbelief'''. IT should '''ONLY''' change the '''CNAME record'''. The '''A records''' should '''NEVER''' change. Even if only one of the instruments is being moved to another dome, BOTH records must be switched or DNS resolution will be intermittent. Example of a switched DNS setup: '''Instrument switched setup'''{| class="wikitable" border="1"|-! Name! Type! Content! TTL|-| shocnawe.suth.saao.ac.za| A| 10.2.2.89| 86400|-| shocndisbelief.suth.saao.ac.za| A| 10.2.2.88| 86400|-| shoc40in.suth.saao.ac.za| CNAME| shocnawe.suth.saao.ac.za| 360|-| shoclesedi.suth.saao.ac.za| CNAME| shocndisbelief.suth.saao.ac.za| 360|} Both '''shocndisbelief''' and '''shocnawe''' machines move between the telescopes. The setup below is what needs to change when '''shocnawe''' is moved. When e.g. the '''shocnawe''' machine is moved from the '''40inch''' to the '''74inch''', '''only''' change the '''40''' to '''74''' like below: '''From'''{| class="wikitable" border="1"|-! Name! Type! Content! TTL|-| shoc40in.suth.saao.ac.za| CNAME| shocnawe.suth.saao.ac.za| 360|} '''To'''{| class="wikitable" border="1"|-! Name! Type! Content! TTL|-| shoc74in.suth.saao.ac.za| CNAME| shocnawe.suth.saao.ac.za| 360|} '''When you ssh after the CNAME change''' Note: This change will result in a warning if someone tried to SSH into the machine - this is safe to accept. ssh ccd@shoc40in.suth.saao.ac.za Warning: the ECDSA host key for 'shoc40in.suth.saao.ac.za' differs from the key for the IP address '10.2.2.89' Offending key for IP in /Users/simon/.ssh/known_hosts:283 Matching host key in /Users/simon/.ssh/known_hosts:317 Are you sure you want to continue connecting (yes/no)? yes '''Possible CNAME changes''' With the CNAME setup below, both the shocndisbelief and shocnawe can be connected in Lesedi. Please ask the IT standby person to help you with this DNS setup. Either SHOC instrument may be mounted on any of the three telescopes. '''How to configure'''{| class="wikitable" border="1"|-! Name! Type! Content! TTL|-| shoclesedi.suth.saao.ac.za| CNAME| shocndisbelief.suth.saao.ac.za| 360|} '''Can be added if required'''{| class="wikitable" border="1"|-! Name! Type! Content! TTL|-| shoclesedi.suth.saao.ac.za| CNAME| shocnawe.suth.saao.ac.za| 360|} ==Run the shocboxswitch.sh script when SHOC is moved between telescopes== Run the script shocboxswitch.sh. The default option reverts to the standard mapping between SHOC computers and telescopes. If the default is not chosen, you'll be prompted for the name of the telescope the SHOC computer is being mounted on.* Check the log entry to make sure the configuration was changed to the telescope you chose. In a terminal type: less /var/log/shoc-camera.log [[image:Camera-log.png|500px]] * Check and make sure that the configuration file points to the telescope name that you chose. In a terminal type: less /etc/init/shoc-camera.conf * Take test data to make sure the data is written to '''/data/<telescope_name>/<instrument_name>/YYYY/MMDD/'''.
==Unmounting SHOC==
** Check that the antenna cable has been connected to the SHOC GPS on the box.
** Check that the antenna has access to the sky (i.e. the dome is open)
** Check that the location settings are correct. Click on 'receiver status' in the GPS tab and look at the latitude and longitude. The Sutherland Latitude is ~32 22 S while Cape Town is 33 56S. If the location is not corrrectcorrect, then the GPS will not find satellites and the correct location needs to be updated as described in the Configuring SHOC section.
* Camera is in an inoperable state:
** Restart the services by sshing into the SHOC computer (see Configuring SHOC section):
*** change into directory /home/ccd/programming/shoc /bin and type the following: ./restart_services. It may be neccessary to do this a few times.*** If that does not work, under /home/ccd/programming/shoc /bin type the following to stop and start in separate steps:
::: ./stop_services
::: ./start_services
