Wiki software last updated 28-Jul-2014 - Comments to Ruben Squartini <ruben@auger.org.ar>

List of Variables to be Studied

And the plots to be made and reference spectra FIXME.

This was discussed heavily during the Frascati Meeting Jan06.

Extracted from the file “daq_monitoring_data.txt”.

  • IMHO (HJM) there should be some more work on that page, eventually the too detailed stuff (for instance the tables here) should be moved to separate pages. FIXME
  • Here and on the follow-up pages we have still a number of things to be defined or clarified, please look for the ?.
  • A critical remark (HJM) concerning references to other tables:
    • they make more work and cost sometimes also time (see T3Tab)
    • if they are still required by somebody, they are important
    • if they are important, procedures must be established to ensure that they are correct (i.e. DB table integrity must be checked)
  • SR: I suggest we use `TableNameId` for the primary keys - this way “natural joins” work naturally :-)

Table Headings

source:
  • 1=calibration files
  • FIXME all the other numbers are not documented!
Ref:
  • R = with comparison to reference histogram
E/M:
  • E=eye-wise
  • M=mirror-wise
  • O=other

GENERAL

For all variables the data type has to be defined!

  • number
    1. telescope number: 1..24
    2. eye number 1..4 (or 5?)
    3. run number : 1…inf
  • string: Comment, some thing undefined
  • emnumerate:
    • status (OK, No connection, Time error, …)

(using enumerate helps to avoid typing errors)

Naming conventions:

These are the naming conventions I (HJM) am using. I ask everybody to follow these recommandations, just to ease the use later

  • table names:
    • fragments starting with capital letters, rest small letters
    • ending with Tab
    • example: RunInfoTab
  • variable names
    • fragments starting with capital letters, rest small letters
    • acronyms might be fully capitalized, for instance: GPS or RMS
    • ids are named Id (auto incremented fields)
    • avoid underscores
    • references to other tables: TableNameId, for instance: RunTabId in table RunInfoTab references Id in RunTab
    • example(s): EventId, GPSSec
  • PRIMARY KEYs and foreign keys:
    • SR: this is a new convention - not yet used everywhere
    • should have the same name in both tables (so that “natural joins” work naturally)
    • (foreign keys are where a table has an attribute, which refers to the primary key of another table)
    • PRIMARY KEYs should be named TableNameId (where TableName is the name of the table)
  • more …

Run Control Information (1)

  • Written at start/stop of RUN
  • Filled at Eye level
  • Name of table: RunTab
  • Table structure and field names

Run Control Information (2)

  • Written at end of DAQ RUNs
  • Filled at Eye level (by the FD Event Builder)
  • Name of table: RunInfoTab
  • Table structure and field names

Run Control Information (3)

  • this is a set of tables used to log the software and hardware configuration, it hasn't to do very much with the monitoring, but I added this info just for completeness
  • Filled at Eye and Mirror level
  • Table names:
  • Table definitions and field names
  • a few more tables will be added in the future (Veto and Datasink settings, for instance)

Mirror DAQ (TLT) Information

  • Written with each TLT event (each SLT event ?)
  • Filled at Mirror level
  • Name of table: T2Tab
  • Table structure and field names

Event Builder (1) (including T3)

  • Written with every SLT/TLT/T3 event ??? (to be defined !!!)
  • Runs at Eye level (Event Builder)
  • Name of table: T3Tab
  • Table structure and field names

Event Builder (2)

  • Written periodically (approx. every 10 seconds)
  • Runs at Eye level (Event Builder)
  • Name of table: EvbStatTab
  • Table structure and field names
  • Table contents will be erased after the shift ends

Central FD

  • Written with every event
  • Runs at Gina / CDAS
  • Name of table: ???

(id)
[event builder id eye 1]
[event builder id eye 2]
[event builder id eye 3]
[event builder id eye 4]

DAQ and Trigger summary

Name Type Variables source Ref. E/M
DAQ status table/histomirrors attached M
connection to CDAS M
connection to EVB M
GPS status
GPS Times.Vetos E
(ext. triggers, bursts..) E
deadtime? M
Slow Control Summaryhisto/table (3) M
time since last eventStop-watchGPS (2) R E
SLT rate history histo(GPS, SLT) (2) R E+M
TLT rate historyhisto(GPS, TLT)(2)RE+M
T3 rate historyhisto(GPS, T3)(2)RE+M
mirror number/T3 triggerhisto(telescopeId, T3 rate)(2,2)R
TLT classification of acc./rej.histo6(classId, rate)(2,2)RE
T3 classification of acc./rej. histo(classId, rate) (2,2)RE
$\Delta t$ of consecutive SLT eventshisto (diff. scale)(\Delta t)(2)RE+M
$\Delta t$ of consecutive T3 eventshisto (diff. scale)(\Delta t)(2)RE+M
nanotime at ground T3histo (GPSns)(2)RE
nanotime at ground TLThisto(GPSns)(2)RE
\phi,,SDP,, for T3 histo(\phi,,SDP,,)(2?)RE
Number of T2 Pixelhisto(\sum Pixel,,T2,,)(2)RE+M
Number of T3 Pixelhisto(\sum Pixel,,T3,,)(2)RE+M
history of shower cand. rate 1)histo(GPS)(2?)RE
av. signal per pixel for shower candidateshisto(\frac{\sum integral I}{number of Pixels)}(2?)RM

bgrecorder/bgloop

Write every 30secs
Runs at Telescope / Eye ?

[run control id] - (read run number of active run from data base?!)
run id
variance matrix / mean variance?
pedestal matrix
hitrate matrix
threshold matrix
telescope id
list of connected telescopes
list of noisy pixel?

better:

  • Written every 30sec
  • per data set 440 lines will be added
  • Runs at telescope stores in eye data base

(id)
telescope id 1..24 uint (5bit)
pixel number 1..480 uint (9bit)
time stamp (sec) uint (32bit)
variance *1000 0-MAX_UINT uint (min 32bit) / typical ~1000 - 50000
pedestal *1000 0-MAX_UINT uint (min 16bit) / typ 150k
/ pedestal = sum / samples + offset
sum 0..MAX_INT-24Bit int (24bit) / not Implemented
/ Attention: Needs samples + offfset !!
hitrate 0..1023 uint (10bit)
threshold 0..16667 uint (14bit)

  • plots/tables when shutter closed/before DAQ
NameTypeVariablessourceRef.E/M
Signal variance vs pixelId1D/2D-Camera histo(Pixelid, Variance)(4)RM
Pixel variance distribution1D/2D histo(Variances)(4)RM
  • plots/tables when shutter open/while DAQ
NameTypeVariablessourceRef.E/M
Signal variance vs pixelId1D/2D histo(Pixelid, Variance)(5)RM
Pixel variance distribution1D/2D histo(Variances)(5)RM
history of average camera variancehisto(GPS, mean variance)(5,5)RM
T1 pixel rate 1D/2D histoPixelid,hit rate (5,5)RM
thresholds 1D/2D histoPixelid,threshold (5,5)RM
pedestals 1D/2D histoPixelid,pedestal (5,5)RM
  • A list of malfunctioning pixels should also be provided.
  • Written periodically when FE crates are on
  • Filled for each FE crate
  • There are a number of sensors attached to each FE crate which measure:
    • 3 temperature sensors (= ADCs)
    • 5 sensors to measure the supply voltages (-5.0, +5.0, +2.5, +3.3, +5.0 V)
  • Should create an ALARM when values are outside pre-defined margins

Error List

  • Written each time an error message above a certain (configurable) level is written
  • Filled at Eye and Mirror level
  • Name of table: ErrorTab
  • Table structure and field names
  • List of messages to be found in the ErrorTab

eg.:
second counter status vector (chktime) of gpsclock, fd electronic, mirrorpcs

Other dependend data to save access time?

Last events

  • Written with every event
  • Filled at Eye``PC level
  • Name of table: LastEventTab
  • Table structure and field names
  • Note: This is a table with only one data set

that is continously overwriten.

Data

  • to be stored periodical or whenever a change occurs:
  • Table structures and field names

Alarms

  • whenever the system takes some automatic action (e.g. due to wind, rain, …) (see here)
  • when a hardware error is detected (e.g. fuses burnt, power loss, low UPS batteries, …)
  • when the SY1527 HV system signals a problem (e.g. overcurrent, undervoltage, fan-fail, …)

Alarms will be written directly into AlarmTab on eye-level

  • Written with every calibration RUN
NameTypeVariablessourceRefE/M
Signal integral vs. pixel idHisto(Pixelid, sum ADC)(1,1)RM
Variance of time bins before signal vs PixelidHisto(Pixelid, Variance of time bins)(1,1)RM
Pixel signal average timeHisto&(Pixelid, mean of pixel times)(1,1)RM
History of light source shots (from diode)Histo(time of shot,intensity)(?,?)RM

Plots of interest:

(id)
list of noisy pixel
list of pixel with strange timing

list of dead pixel

mean signal over all calibration events matrix / per pixel
variance of pedestal matrix / per pixel
pixel signal average time - matrix

led intensity vector

Calibration Information/Overview

  • This table will contain general information and is written for each telescope (1…24) each time the full calibration sequence (fde_test, cal A/B/C) is complete
  • Filled at Eye level
  • Names of tables: CalHWInfoTab, CalAInfoTab, CalBInfoTab, CalCInfoTab
  • Table structures and field names

Calibration Pixel Data

  • This table will contain pixel specific information and is written for each pixel each time the full calibration (fde_test, cal A/B/C) is complete
  • Filled at Eye level
  • Names of tables: CalHWPixelTab, CalAPixelTab, CalBPixelTab, CalCPixelTab
  • Table structures and field names

Definition of Calibration Analysis Error Codes

  • This is a static table containing the definitions of the error codes in a bit string
  • Name of table: CalErrorCodeTab
  • The ErrorCode in CalHWPixelTab, CalAPixelTab, CalBPixelTab and CalCPixelTab can be masked using bit operations and this table
  • Table structure and field names
NameTypeVariablessourceRef.
history of shotshistoGPSSec(7)
nanotime of shotshisto GPSns7
history of total signalhistoGPS,\sum ADC(2/8)
history Laser SDPhistoGPS,\phi,,SDP,,(2/8)
history N. Triggered pixelshisto GPS, pixels (2/8)
Number of stereo LasertableeventId,GPS,(?)

By parsing the autolog files we get the following parameters from the 'autolog' files:

NameTypeVariablessourceRef.
CLF IdintegerCLFId
Shot IdintegerShotId
GPS timeintegerGPSSec
GPS nanoseconsintegerGPSns
relative energy 1floatRelEn1
relative energy 2floatRelEn2
azimuth anglefloatAzimuth
zenith anglefloatZenith

Data synchronization:

  • Every 5 minutes a cronjob run by monitor@moni gets the most recent data and puts it into the database
  • Name of database: AugerMonitor
  • Name of table: CLFtab

Data synchronization:

  • is done in the same way as for the CLF

Stereo / Hybrid

NameTypeVariablessourceRef.
history \Delta t between CELESTE and CLFhisto GPS, tground_FD-tcleste(9)R
hybrid ratestop watch(GPS Hybrids)(9)
history of golden hybrid ratehisto(GPS Hybrids)(9)
history of \Delta tFD-SD from hybridshisto(GPS, tground_FD-tcore_SD)(9)
history of \Delta tFD-SD from golden hybridshisto(GPS,tground_FD-tcore_SD))(9)
history of FD pixels of Hybridshisto(GPS, number of pixels)(9/2)
history of FD pixels of golden Hybridshisto (GPS, number of pixels)(2/9)

Miscellaneous

  • Try: number of PMTs more than 4 sigma from the mean parameters (e.g. hitrate)

Table Optimisation

There are lots of tricks in the manual to optimise the speed of the queries in the PHP webpages. One thing that we should try to do is avoid using variable length atributes (columns), i.e. no VARCHAR, TEXT or BLOB. If a (MyISAM) table contains no variable length atributes then the engine is able to optimise it efficiently on the disk. MyISAM uses a different storage scheme for tables with variable length atributes.

I (Julio) think that it would be useful to save ALL parameters measured by the weather stations in each eye every 5-10 minutes to the database. I don't know how this should be done, I am just trying to open the discussion…FIXME

1)
define shower candidate!

Log In