Verification of Particle Listings

Overview

All entries added to the Particle Listings in the Review of Particle Physics need to be verified by the authors of the publication from which the corresponding results were extracted. Within PDG, such new entries are referred to as "encodings".

In the past this verification was done by e-mailing proofs of new entries to a contact person for the publication. This "verifier" might be the physics coordinator of the corresponding experiment or a contact person identified in the publication as the corresponding author.

Starting with the updates for the 2018 Review of Particle Physics, a new web-based system (http://pdgverify.lbl.gov) is used. The new system is open (everyone can see what PDG is currently adding to the Listings) but requires an authorization code to suggest any changes. Requests to verify new entries are e-mailed weekly to the corresponding verifiers, providing a link to the list of papers whose entries should be verified as well as the authorization code to request corrections.

Physics areas

In order to group papers to be verified in a meaningful way, PDG assigns each paper to one or more so-called physics areas. These physics areas are:

  • gamma
  • gluon
  • W, Z
  • SM Higgs
  • Searches for Higgs Boson
  • Searches for other bosons (W`, Z`, Axions)
  • e, mu, tau
  • Neutrino
  • Searches for heavy leptons
  • Quarks (excluding top)
  • Top quark
  • Quark searches (b`, t`, free quark)
  • Light mesons (pions, eta)
  • Strange and charmed mesons
  • B mesons
  • Non qqbar candidates
  • Baryons
  • Supersymmetry
  • Other searches (excluding supersymmetry)
  • Unstable mesons, charmonium and bottomonium
  • graviton
In the past all verification requests were sent to a single verifier in each experiment. The new verification system allows to distribute verification requests to different persons based on these areas. If your experiment would like to make use of this possibility, please contact support@pdg.lbl.gov and indicate who should receive verification requests for specific physics areas. Verification requests in physics areas where no specific verifier is assigned will be sent to the main verifier for that experiment.

Instructions for verifiers

When new entries are ready to be verified, the corresponding verifier will receive an e-mail requesting verification (see sample e-mail). Such e-mails are sent once a week. Except when PDG is finalizing the Listings and any remaining verifications must be submitted immediately, e-mails are only sent when new entries have become available since the last e-mail was sent.

When receiving an e-mail requesting verification, please

  1. Respond within one week by going to the URL indicated in the e-mail. You will see a list of all the papers whose entries you are asked to verify (see example), grouped by physics areas. Expand each area (by clicking on its name) to see the corresponding papers to verify.
  2. For each paper where the status is "to be verified", please click on the "check" button, then:
    • If this is the first time you are verifying PDG data (or if you used a different browser or cleared stored cookies), you will be asked to enter your name, e-mail address, and possibly the authorization code (the authorization code will be provided automatically if you use the link in the e-mail). See example.
    • Follow the instructions on the verification page (see example): check all entries marked in yellow, if needed suggest corrections, and finally submit your feedback using the buttons at the bottom of the page.
    • If you request changes, you will receive a copy of your requests by e-mail.
  3. After your initial submission, you may update your response (click on the "revise feedback" button).
  4. If you request corrections, the corresponding PDG overseer will look at your suggestions and implement them if he or she agrees. Once this is done, the status of the corresponding encoding will change from "checked - to correct" to "revised". You may (but do not have to) re-verify the updated entry and indicate that it is now correct, or you can request further changes. You may be contacted via e-mail by the corresponding PDG overseer to request clarification, especially in case the overseer should disagree with your suggestion.

Instructions for overseers

With the new verification system, encodings are verified as soon as the corresponding overseer signs off a new encoding. Unless you request (using the messaging system in PdgWorkspace) that the editor checks the encoding before you sign it off, he will not have looked at the encoding before it is sent out for verification. It is therefore entirely your responsibility to make sure that the entry is correct and displays correctly on the signoff page in PdgWorkspace.

When you sign off an encoding, the corresponding verifier will be notified automatically by e-mail the next time verification requests are sent (this is done once a week). The verifier will be asked to verify the encoding within one week from the time he receives the e-mail. Your task list in PdgWorkspace displays the current verification status of each encoding.

If the verifier requests any corrections:

  • You will receive an e-mail with the requested corrections (see example).
  • The corresponding encoding will be returned to your list of pending tasks in PdgWorkspace with a verification status that indicates that corrections are needed.
  • A copy of the submitted comments will be available as a new message for that encoding in PdgWorkspace. The message has a link to a page that shows the submitted comments in the context of the encoding. This link is also provided in the e-mail.
  • You should go to PdgWorkspace and, if you agree with the requested corrections, update the encoding as soon as possible. When you are done, you need to sign off the revised encoding. You need to do this even if you disagree with the verifier and decide not to make any changes. Otherwise the encoding will remain in your list of pending tasks.
  • If you decide to not implement a requested correction, please contact the corresponding verifier via e-mail to explain your decision.

You will not get an e-mail notification if the verifier agrees with your encoding. You can see from the task list in PdgWorkspace (see "Completed Tasks") whether an encoding was verified.

Details of verification workflow

The new verification system is based on the following workflow:

  1. When a paper is added in the PDG system to the list of papers to look at, the paper initially appears in the verification system in the list of ""Papers where PDG has not yet assigned experiment/verifier".
  2. Once the PDG encoder or overseer responsible for that paper determines the verifier, the paper moves to the list for the corresponding verifier/experiment. In the verification system, the paper is now shown with a status of "in preparation".
  3. When the PDG overseer determines that a draft encoding for a paper is complete and "signs off" on the encoding, the paper/encoding is shown in the verification system with a status of "to be verified" and a "check" button appears next to it in the list. (Side note: If multiple overseers look at the same paper for results in different areas, there may be multiple encodings for the same paper that need to be verified independently. Thus a given paper may show up multiple times in the list of papers to be verified.)
  4. When the weekly requests for verification are e-mailed the next time, the new encoding is included and the corresponding verifier will get an e-mail requesting verification (see sample e-mail). Verification should be done within one week from receipt of the request.
  5. The verifier should click on the check button for each encoding in his or her list, leading to the verification page for that encoding The first time he/she will be asked to enter name, e-mail address and possibly authorization code (see example). This information will be stored in the verifier's browser and doesn't need to be re-entered.
  6. On the verification page for a given encoding (see example), the verifier can indicate either that the encoding as proposed is correct, or he/she can suggest changes to each of the new measurement values or footnotes. If changes are suggested, the verifier will receive a copy of the suggested changes by e-mail, and the status of the encoding will change to "checked - to correct". If no changes are needed, the status of the encoding will become "checked - OK".
  7. If the verifier requests changes, the corresponding PDG overseer will be notified by e-mail. After making any necessary changes, the overseer needs to sign off the encoding again. If the overseer decides not to implement the suggested changes, the encoding still needs to be signed off.
  8. When the overseer signs off the encoding after implementing any necessary changes, the status of the encoding changes to "revised". At this time the verifier may again check the encoding and indicate that it is now OK, or suggest further changes. In the latter case, the verification cycle resumes with the overseer being notified of the request for changes, etc. Re-checking of revised encodings by the verifier is appreciated, but not necessary.

Examples

The screen shots below illustrate each step of the verification process.

Sample e-mail requesting verification of new entries

Requests to verify new entries are sent by e-mail to the corresponding verifiers. Usually these e-mails are sent once per week and only if there are new encodings to be verified. Here is an example of such an e-mail:

reminder.png

List of papers to verify

In the e-mail requesting verification of new entries, the verifier is directed to a page showing all papers to be verified, grouped by physics area. Here is an example:

1summary.png


By clicking on a physics area, the list of papers to verify in this area is shown:

2summary.png

Entering verifier information

The first time a verifier checks the entry for a paper, he/she needs to enter name, e-mail address and possibly the authorization code needed to request corrections (the latter is automatically provided when using the link in the e-mail). This information will be stored in cookies in the user's browser and usually doesn't need to be reentered (except when the cookies are deleted, e.g. after restarting the browser, or when using a different computer). The authorization code is given in the e-mail requesting verification and will be the same for the verification of all papers from a given experiment, even if there are multiple verifiers. This allows the physics coordinator to forward verification requests as needed without having to provide different authorization codes.

Having the name and e-mail address of the actual person verifying an entry is important to allow any followup questions from PDG to clarify requests for corrections. It also allows to send a copy of the response by e-mail to the person submitting it.

3ver.png

Verification of a paper

On the verification page for a given paper, i.e. the page reached when clicking on the "check" or "revise feedback" button, all entries to be verified are shown in yellow. Comments and requests for corrections should be added in the corresponding yellow area by clicking on the pencil icon. This will open a text window to submit comments or corrections. A simple editor provides some formatting tools as well as a formula editor supporting LaTeX to enter symbols and equations.

In addition, a yellow area is provided near the bottom of the page to provide general comments.

When making comments or requesting corrections, PDG-internal identifiers such as the NODE, OCCUR or LINKAGE references given in the right border can be used to refer to specific items on the page.

4ver.png

5ver.png

Clicking the pencil icon expands a text field:

6ver.png

The "fx" icon in the toolbar of the text field provides a math editor where formulas and symbols can be entered using LaTeX:

v06-verify4.png

Responding to PDG

After verifying that the new entries for a paper, including author(s) and collaboration, measurements, limits, as well as any footnotes are correct and, if necessary, providing suggestions or requesting corrections, the verification feedback is submitted to PDG using either of two buttons at the end of the page:

  • "This entry is OK - no changes needed": The response that everything is OK will be recorded in the PDG database. No e-mail notifications will be sent.
  • "Submit corrections and send response to PDG": To use this button, at least one comment or suggestion for changes must have been entered. This button should be used whenever any comments are submitted, even if no corrections are requested. Upon clicking this button, an e-mail with the response will be sent to the corresponding PDG overseer. You and the verifier for this paper (if different) will be CCed.

7ver.png

List of verified papers

If a paper was checked, it will be shown in the list of papers with a status of

  • "checked - OK" if no corrections are needed
  • "checked - to corrrect" if corrections were requested but have not yet been implemented by the overseer
  • "revised" - if corrections were requested and dealt with by the overseer (this status does not imply that the overseer implemented all requests/suggestions)

After the initial verification, the "check" button changes to "revise feedback" and can be used to submit any further comments or suggestions.

8summary.png

Sample e-mail summarizing requested corrections

When comments or requests for corrections are submitted to PDG, an e-mail with all suggestions is sent to the corresponding PDG overseer, with CC to the person submitting the comments and to the verifier for the paper (if different from the person submitting the comments). An example of such an e-mail is shown below.

response4.png

-- No permission to view TWiki.UserReports -- No permission to view TWiki.UserReports

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng 1summary.png r1 manage 181.4 K 2017-09-26 - 23:30 Kirill  
PNGpng 2summary.png r1 manage 108.0 K 2017-09-26 - 23:30 Kirill  
PNGpng 3ver.png r1 manage 37.2 K 2017-09-26 - 23:30 Kirill  
PNGpng 4ver.png r1 manage 134.7 K 2017-09-26 - 23:30 Kirill  
PNGpng 5ver.png r1 manage 140.5 K 2017-09-26 - 23:30 Kirill  
PNGpng 6ver.png r1 manage 26.6 K 2017-09-26 - 23:30 Kirill  
PNGpng 7ver.png r1 manage 9.8 K 2017-09-26 - 23:30 Kirill  
PNGpng 8summary.png r1 manage 72.3 K 2017-09-26 - 23:30 Kirill  
PNGpng mail1.png r1 manage 40.9 K 2017-09-14 - 23:58 Kirill  
PNGpng mail2.png r1 manage 39.1 K 2017-09-14 - 23:58 Kirill  
PNGpng mail3.png r1 manage 36.5 K 2017-09-15 - 00:07 Kirill  
PNGpng reminder.png r1 manage 40.0 K 2017-09-27 - 22:34 Kirill  
PNGpng response.png r1 manage 39.7 K 2017-09-27 - 22:34 Kirill  
PNGpng response3.png r2 r1 manage 38.7 K 2017-09-28 - 20:05 Kirill  
PNGpng response4.png r1 manage 38.7 K 2017-09-28 - 20:06 Kirill  
PNGpng summary1.png r1 manage 174.1 K 2017-09-14 - 21:38 Kirill  
PNGpng summary2.png r1 manage 131.9 K 2017-09-14 - 21:38 Kirill  
PNGpng v01-summary1.png r1 manage 128.2 K 2017-08-30 - 20:53 Kirill  
PNGpng v02-summary2.png r1 manage 110.0 K 2017-08-30 - 20:54 Kirill  
PNGpng v03-verify1.png r1 manage 99.9 K 2017-08-30 - 20:54 Kirill  
PNGpng v04-verify2.png r1 manage 165.0 K 2017-08-30 - 20:54 Kirill  
PNGpng v05-verify3.png r1 manage 44.0 K 2017-08-30 - 20:54 Kirill  
PNGpng v06-verify4.png r1 manage 71.4 K 2017-08-30 - 20:54 Kirill  
PNGpng v07-verify5.png r1 manage 58.5 K 2017-08-30 - 20:54 Kirill  
PNGpng v08-summary3.png r1 manage 69.9 K 2017-08-30 - 20:54 Kirill  
PNGpng ver1.png r1 manage 53.1 K 2017-09-14 - 21:38 Kirill  
PNGpng ver2.png r1 manage 165.6 K 2017-09-14 - 21:38 Kirill  
PNGpng ver3.png r1 manage 48.8 K 2017-09-14 - 21:38 Kirill  
PNGpng ver4.png r1 manage 8.4 K 2017-09-14 - 21:38 Kirill  
PNGpng ver5.png r1 manage 70.2 K 2017-09-14 - 21:39 Kirill  

This topic: Pdg > WebHome > Verification
Topic revision: r21 - 2019-02-28 - Beringer
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright Information: This page and all following are copyrighted by the Regents of the University of California