Changes

Jump to: navigation, search

SHOC

3,419 bytes added, 07:38, 18 June 2018
/* Automated Data Acquisition (Scripting) */
*'' If you have found this webpage already loaded on the browser, please refresh to ensure that it is current!!''
== SHOC Call Out list ==
'''Instrument software issues:''' Carel van Gend 076 818 1312**050 (0768181312)
'''Telescope software issues:''' standby **113 (0829034869) or 9118
Dr. Marissa Kotze has written some [http://shoc.saao.ac.za/Pipeline/ pipeline] software which may help with reduction of SHOC data.
===IMPORTANT NOTES===* '''IMPORTANT NOTES ABOUT THE NEW SOFTWARECONSIDER DECREASING DETECTOR TEMPERATURE IF TAKING LONG EXPOSURES''' Reports in 2016 indicated that the dark current is higher than expected (by a factor of ~ 100; more information to follow). This is most noticeable for long (>=10 minute) exposures. The camera is automatically cooled to -50C. It can be cooled down to -70 C; however, if the ambient temperature is high, the cooler will be stressed and cannot reach minimum temperatures. If you do adjust the temperature to -60, -65 or -70, please listen carefully so that if the temperature alarm triggers (a high-pitched, continuous beep) you can stop cooling and prevent hardware failure. March 2017* '''THE NEW SOFTWARE HAS DIFFERENT FILE NAMES AND HEADER KEYWORDS THAN PREVIOUS DATA.''' If you use this information in your analyses, you will have to update it. The filename format has been standardized to <Instrument>_YYYYMMDD.####.fits, where the instrument is sha (shocnawe), shd (shocndisbelief), or SHH to designate which SHOC box is being used. Please check the header of a new data file or the wiki [http://topswiki.saao.ac.za/index.php/SHOC#FITS_Header_Keywords FITS header keywords] (the "new" keywords are directly from the Andor iXon camera under Linux). March 2015* '''THE FILE SIZE IS NO LONGER LIMITED TO 2 GB. ''' This means that if you take large data cubes, the file size has ''no limit'' (this is because we are no longer on a Window-based, 32-bit, operating system). If you end up with a very large file, data analysis can be difficult. We are working on an efficient method to automatically split files into smaller sizes, but for now users need to be aware that requesting a large number of images in one file could return a very large file size (and if you mistakenly delete your file, you have lost alot of work!). It is recommended that multiple files be manually taken. ''If you have any questions or problems with the data, feel free to contact carel@ saao.ac.za or amanda@ saao.ac.za.''March 2015
== Change Log ==
'''Most recent chances changes at the top'''* '''v1.4.2 Release date 3 January 2018<br>'''** Feature: Better error reporting** Feature: TCS info now available when mounted on the 40-inch * '''v1.4.0 Release date 23 November 2016<br>'''** Bug: Camera interface doesn't revert to "manual mode" when reloading the page while it's in scripting mode** Feature: Upgraded to JS9 v1.10 ([https://github.com/ericmandel/js9/releases/tag/v1.10 Release Notes])** Feature: Display and plot total counts for active regions on image display * '''v1.3.5 Release date 16 November 2016<br>'''** Feature: GPS driver now has access to a general-purpose SHOC config file** Feature: GPS antenna delay is now set from the config file  * '''v1.3.4 Release date 19 October 2016<br>'''** Bug: Default image display orientation. The image shown on-screen in conventional mode should now be correctly oriented.
* '''v1.3.3 Release date 17 August 2016<br>'''
When running a series of exposures on the same target with different filters it
can be useful to make use of the automated acquisition mode on the camera tab.'''NOTE: SCRIPTING ALWAYS USES THE INTERNAL TRIGGER'''
[[File:scripting_1.png|1000px]]
When you click the "New script" option a dialog will appear giving you
<span style="color: red;">'''(1)'''</span> a text field to enter a name for your new script. You can then enter any
name and click the <span style="color: red;">'''(2)'''</span> "Create Script" button to create a new one. ''NOTE: Please ensure that your script has an easily identifiable name for the observer and run, e.g. Gulbis_20150617Script1Sickafoose_20150617Script1. All scripts are currently being stored in the drop-down web interface -- this means you will see other people's scripts and will have to look there for your own in order to load the script in subsequent observing runs.''
==Data Transfer==
There is an automatic transfer of data from the instrument to the file servers in Sutherland and Cape Town, every morning at 7:30. So you should not need to worry about the security of your data. However, you'll still need to access your data! We describe here how to do this. There are two ways to transfer get the data: (1) directly to your computer from the instrument PC or (2) onto from the SAAO storage area network (SAN). The data are automatically copied to the server every morning (at 7:30am).* To use method (1) To transfer data directly to your computer, use the command line.:
** Open a shell on your local PC (Putty if you're using Windows or a bash/tcsh/sh if on Mac or Linux)
** Copy the data using the following command, depending on the telescope:
rsync -avzP shoc40in@shoc40in.suth.saao.ac.za:"/data/40in/shd/2016/0503/*.fits" /LocalFilePath
* (2) To move data to the SAN on the Sutherland plateau (from which you can access it by connecting to the SAAO server), there is a data copy script on the SHOC computer. ** From a command line, connect to the instrument computer. As appropriate for your telescope, use either of the following commands: ssh shoc74in@shoc74in.suth.saao.ac.za ssh shoc40in@shoc40in.suth.saao.ac.za ---- When prompted for the password, use the password for the 74in or 40in user.**Run the shocdatacopy.sh script. Options can be: shocdatacopy.sh -d yesterday # to transfer yesterday's data shocdatacopy.sh -d YYYYMMDD # to transfer data from some specified date ---- Without parameters, it will try to transfer today's data.** To collect your data from the SAN, you'll have to secure copy it from '''astro.suth.saao.ac.za'''. The usernames for this are '''ccd40''' and '''ccd74''' depending on your telescope. Ask for the password. Data are in the path /data/telescopedata/<TELESCOPE>/sh?/<YYYY>/<MMDD>
rsync -avzP ccd74@astro.suth.saao.ac.za:/data/telescopedata/74in/sh?/<YYYY>/<MMDD>/*.fits /LocalFilePath
rsync -avzP ccd40@astro.suth.saao.ac.za:/data/telescopedata/40in/sh?/<YYYY>/<MMDD>/*.fits /LocalFilePath
* Below this, the readout speed and preamp gain may be set.
* Below this, the readout speed and preamp gain may be set.
* The last setting is the Electron Multiplying (EM) gain. <span style="color: red">NOTE: DO NOT USE ELECTRON MULTIPLYING MODE UNLESS YOU HAVE SPECIFICALLY OBTAINED PERMISSION FROM DR AMANDA GULBIS SICKAFOOSE TO DO SO, AND YOU UNDERSTAND HOW AND WHEN TO USE IT. PERMANENT DAMAGE TO THE SENSOR CAN RESULT FROM INCORRECT USAGE. If you are using EM mode, choose the EM output amplifier, and then select a gain. BEGIN WITH LOW GAIN VALUES, BECAUSE IF THERE IS TOO MUCH SIGNAL''' YOU CAN DAMAGE THE CAMERA.'''</span>
* When setting the horizontal shift speed, preamp gain and (if using EM mode, the EM gain), note the following: Higher readout speeds allow for shorter exposure times, but at the expense of noisier data, while higher electron to ADU gain values decrease the dynamic range. Read noise and electron to ADU gain values for each mode are provided in the PDF manual. In addition to this, it should be noted that as it takes about 6 milliseconds for the top-most pixel in the frame to be shifted to the storage area and SHOC does not have a shutter. The top row of pixels in each frame will have an exposure time of about six milliseconds longer than the bottom row. Consequently although it is possible (with enough subframing and binning) to expose for less than a hundredth of a second, this is inadvisable for the above reason. Finally, as indicated above, the 1MHz CON and EM modes are both 16bit. What this means is that the mode will reach saturation at 2^16 = 65536 counts. On the other hand, the 3 MHz CON and EM and 5,10 MHz EM modes are all 14-bit modes and saturate at 2^14 = 16384 counts. As the saturation limits of each mode will also depend on the electron to ADU gain, tables in the printed manual give the saturation limit in both counts and electrons for each mode.
* To get a quick view of the images that will result, press the "Preview" button. Images will be displayed in the left-hand display panel. Press "Stop" when you're done.
(vi) The regions will continue to appear on the display under the "Controls" GUI tab. Selecting "Region" -> "Delete Regions" will remove all of the regions shown on the frame.
[[File:RealTimeSHOC.jpg|800px|left]]<br clear=all>
 
==== Totcounts and bscounts ====
Totcounts are the total counts in the box region, while bscounts has the calculated background (see below) subtracted from the total.
 
==== Background and Noise ====
The background and noise are obtained from the outer 4 pixel-wide perimeter of the selected region. The background value is the median value of this area and the noise value is the RMS.
 
==== Centroid ====
The centroid is obtained by locating the brightest 9 pixel box and computing the centroid of the background subtracted data about this point.
 
==== FWHM ====
The FWHM is estimated from the image moments about the centroid, assuming a Gaussian distribution.
===FITS Header Keywords===
===LED timing tests===
''Currently, this This functionality is only should be available on the 74". Eventually, it should also be on the 40" and the new 1m. It has only been tested on the 74".''The idea is to take a long, GPS-triggered data cube with SHOC at high cadence, while running the LED at 1 PPS (which has a longer pulse on the minute and can thus be used to test absolute timing accuracy).*The dome and mirror covers must be OPEN for this test to work. (The GPS needs to get satellite signal, and the LED must be seen through the light path.)
*There is an LED mounted in the telescope optical path that can be used to independently verify SHOC timing. Log into the following website:
[[image:74inLEDPulseWebsite.jpg|500px]]
*Instructions Check that the LED is visible by turning on what data a pulse and seeing if it is apparent in the preview images. For example, take 0.2-s exposure and set the LED to 1 PPS, then look for increased counts/flashes on the display every 5th frame.* '''To take a timing data cube:''' ** start the LED at 1 PPS** in preview mode, increase the SHOC binning and how readout speed to reach the desired exposure cadence (< 0.05s). ** set SHOC to external timing, select >=10000 frames, and start the GPS with repeated pulses at the chosen cadence ** REMEMBER TO TURN OFF THE LED PULSES WHEN DONE.* '''Analysis:''' you are free to analyze your own timing data! Otherwise, an analysis pipeline has been written by Marissa Kotze and is supported by Carel van Gend. Carel can assist with data analysis.* Download the SHOCpipeline.tar and PULSEtiming.tar software. Create a directory for the software (e.g. timingtests) and extract the software to that directory: mkdir timingtests cd timingtests tar -xvf ../SHOCpipeline.tar tar -xvf ../PULSEtiming.tar * In the README file, follow the instructions on setting up iraf. * Then make a further subdirectory for the testdata: mkdir testdata * Copy the data cube to this directory, and run the testPULSEtiming.py script: cd testdata python testPULSEtiming.py <SHOC fits file name> * The software will be posted soonexamine the header, ask the user for confirmation of a few values, and then verify the timing. A report is printed out at the end.
== In-depth Information ==
To use the command line interface:
** Fire up a terminal
** $ ssh <user>@<shocndisbelief|shocnawe>.suth.saao.ac.za** $ cd /home/ccd/programming/shoc-gps** $ python -cli.py
** The commands in the cli are help, status, pop, mask, bias, timing_mode and location. Use "help <command>" for more information.
** The timing_mode option makes the GPS use static or dynamic mode.
==Switching the SHOC computer to a new telescope==
Notify the IT standby person, so that the DNS records can be altered - they should use the information below.
'''Normal Setup'''
|}
The shoc computer should be reachable by an '''alias ''' containing the telescope name. Usually '''shoc74in ''' is an alias to '''shocnawe''', and '''shoc40in ''' is an alias to '''shocndisbelief'''. If, e.g. '''shocndisbelief ''' is mounted on the 74 inch '''74inch''' telescope, the alias '''shoc74in ''' should be changed to point to '''shocndisbelief'''.
IT should '''ONLY''' change the '''CNAME record'''. The '''A records''' should '''NEVER''' change. Even if only one of the instruments is being moved to another dome, BOTH records must be switched or DNS resolution will be intermittent. Example of a switched DNS setup:
'''Instrument switched setup'''
{| class="wikitable" border="1"
|-
|}
The '''above''' setup was always the '''normal''' setup but now the shocndisbelief machine is in the 1m dome (old 30Inch). The '''shocndisbelief''' will no longer be in the 40inch and will not move around, only the '''shocnawe''' machine will move between the telescopes. Therefore the setup below is what needs to change when the '''shocnawe''' is moved.
Note that the '''shocnawe''' now points to '''shoc40in.suth.saao.ac.za'''. If it is configured like this, that means the '''shocnawe''' machine is in the '''40inch'''. When it is moved, '''only''' change the '''40''' to '''74''' like below:
'''From'''
'''Possible CNAME Changes'''
  With the CNAME setup below, both the shocndisbelief and shocnawe can be connected in the new '''1m'''. Please ask the IT standby person to help you with this DNS setup. At this moment '''only''' the '''shocnawe''' will be moved between the 40inch and 74inch. The '''shocndisbelief''' is in the '''1m''' and will '''not''' be moved between the 40inch and 74inch.
'''How It It Configured Now'''
'''Run Shocboxswitch.sh When Shoc Is Moved Between Domes'''
Run the script shocboxswitch.sh. The default option reverts to the standard mapping between shoc computers and telescopes. If the default is not chosen , you'll be prompted for the name of the telescope the shoc computer is being mounted on.
==Unmounting SHOC==
79
edits