The Graphics and Workstation Performance Group (SPEC/GWPG)
Policies and Procedures Document

Version 1.19
Last Updated: 03/03/2020

1. Graphics and Workstation Performance Group (GWPG)

   The GWPG manages and supervises the development of SPEC’s GWPG benchmarks. Three subcommittees are under the umbrella of the GWPG:

   • Graphics Performance Characterization subcommittee (henceforth SPECgpc^SM)
   • Application Performance Characterization subcommittee (henceforth SPECapc^SM)
   • Workstation Performance Characterization subcommittee (henceforth SPECwpc^SM)
   
   
2. 1.1. Scope of Rules
   
   
   1. The rules contained in this document shall apply to the GWPG and all three GWPG subcommittees: SPECgpc, SPECapc, and SPECwpc.
      
      
   2. Each subcommittee shall maintain its own rules document, which shall apply in conjunction with this document.
      
      
   3. Where a subcommittee’s rule overrides a rule in this document, this will be explicitly indicated in that subcommittee’s rules document.
      
      
   4. Should a new subcommittee be approved or an existing one dissolved, this document shall be updated accordingly.
      
      
   5. In the event of disagreement between this document and SPEC's Bylaws, or disagreement between this document and decisions by the SPEC Board of Directors, said Bylaws or decisions shall take precedence over this document.
   
   
3. 1.2. Joining GWPG

   Membership in the GWPG is open to sustaining member or Associate member. GWPG subcommittees may also include members who are non-voting supporting contributors.

   1. 1.2.1. Sustaining Members

      Sustaining members pay annual dues and accrue all the rights, privileges and responsibilities that full membership entails. Sustaining memberships are available to commercial and non-commercial entities that have direct and/or material interest in graphics or workstation-related application performance benchmarking. Any eligible entity that wishes to join GWPG may join provided they pay annual dues as determined by the Board of Directors.

      Sustaining members may become active participants in any subcommittee of GWPG.

      Sustaining members are eligible to stand for election for the subcommittees, nominate candidates for offices and the Board of Directors, and to champion proposed benchmarks.

      Sustaining members have the opportunity to review and comment on all benchmarks developed by GWPG. This can be done through active participation in the subcommittee developing the benchmark during any phase of development from design to testing to release. This participation is expected but not required.

      Sustaining members are entitled to secure access to benchmark development source code.

      Sustaining members are entitled to unlimited publication rights, including benchmark result submissions for publication on the SPEC website.

   2. 1.2.2. Associates

      To encourage participation from educational and non-profit institutions interested in its work, GWPG maintains an Associate category that enables these organizations to share in the GWPG process, but not to participate in the general member voting.

      Associate status is available to educational and non-profit institutions.

      All rights and rules of SPEC and GWPG apply to the Associate, unless specifically stated otherwise.

      Associates are entitled to access to development source code.

      Associated do not have voting rights.

   3. 1.2.3. Supporting Contributors

      The role of a Supporting Contributor is available by invitation to a commercial organization, an academic institution, or an individual. A subcommittee may issue an invitation to an interested party to apply for this role. The application must be approved by GWPG. Supporting contributors may help develop new benchmarks, provide expertise to the subcommittee or help provide other support within the subcommittee.

   
   
4. 1.3. Membership Dues
   
   
   1. Dues Structure: Dues paid to the GWPG authorize membership in all GWPG subcommittees. Dues for the GWPG are determined by the SPEC Board of Directors with input from the GWPG.
      
      
   2. Dues Payment: Members are expected to remain members in good standing by the prompt remittance of annual dues before the start of the year in question. Membership privileges will be suspended if dues have not been received by the SPEC office by March 31^st. Membership privileges will be restored once dues have been paid.
   
   
5. 1.4. Organization and Voting
   
   
   1. 1.4.1. Officers
      

      Officer election: On an annual basis, at the first face to face meeting of the year, GWPG and the subcommittees will elect the following officers from their eligible voting members: chair, vice-chair, and secretary. Officer positions are held by individuals.

      Conflict of Interest: If a conflict of interest affects an officer in pursuance of his or her duties, and if any defined succession of responsibility fails to resolve the conflict of interest, the committee may appoint a committee member to fulfill the officer’s duties for the scope of the matter in which the conflict of interest exists. For example, the chair is a party in a re-review or appeal occurrence.

      Vacancy: An election will be held to fill a vacancy if an officer steps down, except for the case where the chair steps down and a vice-chair has already been elected; in this case, the vice-chair shall become the chair for the duration of the term, and an election for a new vice-chair must be held to fill the vacancy.

      Chair Responsibilities

      • Conduct meetings
      • Send out an agenda prior to the meetings
      • Conduct votes
      • Manage interactions with outside organizations such as the press or customers
      • Represent and respond on behalf of the group to external question and queries
      • Interact with the GWPG committee
      • Manage the submission, review and appeal process, including assigning members to reviews

      Vice-chair Responsibilities

      • Perform the chair’s role when the chair is unavailable, or if the chair is subject to a conflict of interest.

      Secretary Responsibilities

      • Record notes from all meetings
      • Maintain the subcommittee’s rules document
      • Keep a history of violations
      
      
   2. 1.4.2. Meetings

      GWPG and GWPG subcommittee meetings are conducted in accordance with general parliamentary procedures under direction of the chair.

      Guidelines for meetings:

      • The chair controls the meeting
      • Members participate and aim to reach a consensus
      • Members do not interrupt each other or dominate the proceedings
      • Members keep an open mind and listen to the opinions of others

      GWPG subcommittees have three types of meetings: quarterly face to face meetings, weekly conference call meetings, and special meetings.

      Face to face meetings are the only meetings that are qualified for attendance. A member loses their voting rights upon missing 2 consecutive qualified meetings.

      Face to face meetings are scheduled at least one month in advance. Meetings by conference call are scheduled every week unless otherwise stated due to conflicts that prohibit quorum or holidays. Special meetings must be scheduled with at least one week notice.

      For a qualified meeting for which attendance in person is expected, for example face to face meetings, attending by conference call does not count as qualified attendance.

   3. 1.4.3. Voting
      
      
      1. Voting: Issues may be designated for resolution by casting a vote by eligible voting members of the GWPG or subcommittees. Votes may be made by conference call, in person at a face to face meeting, email, or a combination of the three.
         
         
      2. Voting Process: GWPG drives towards consensus among its participants whenever possible. However to ensure progress, formal member votes will need to be taken. An active member may vote ‘yes’, ‘no’, ‘abstain’, or ‘pass’ when the vote is first called. If the member votes ‘pass’, the chair will return to that member to finalize their vote as either ‘yes’, ‘no’, or ‘abstain’ after the remaining members have voted. Votes for ‘abstain’ count only toward establishing quorum. The reasons for voting ‘no’ or ‘abstain’ shall be recorded in the minutes. Such recording can help with the drive for consensus, as it allows others to respond to the specific concerns of those who did not vote ‘yes’.
         
         
      3. Quorum: A valid vote requires a quorum. A quorum is met if at least 66% of eligible voting members respond, i.e. with 5 voting members, 4 must be present to vote. It is also required that quorum not drop below 3 members.
         
         
      4. Proxy voting: GWPG does not allow use of proxy voting, that is, the authorization of an individual who is not an employee of a member institution to vote on behalf of that member.
         
         
      5. Voting method: GWPG and its subcommittees use the simple-majority voting rule, except for the adoption of new or modified benchmark components, which requires a 2/3-majority vote of GWPG eligible voting members. Simple-majority means the highest number of votes of ‘yes' or 'no' wins the vote. If there is a tie, the motion fails. 2/3-majority means that at least 2/3 (rounded up) of GWPG eligible voting members must vote ‘yes’. If there are not sufficient ‘yes’ votes, the motion fails.
         
         Changing GWPG voting rules requires a 2/3-majority vote of GWPG eligible voting members.
         
         The table that follows details the number of ‘yes’ votes required to pass a 2/3-majority vote:
         
         

         Eligible voting members	3	4	5	6	7	8	9	10	11	12	13	14	15
         ‘Yes’ votes required	2	3	4	4	5	6	6	7	8	8	9	10	10

         
         
      6. New members: New members are eligible for voting rights at the second consecutive qualified quarterly face to face meeting is attended after becoming a member.
         
         
      7. Voting rights and eligibility: A member maintains voting rights by attending at least every other face to face meeting. For a qualified meeting for which attendance in person is expected, attending by conference call does not count as qualified attendance. A member regains voting rights at the second consecutive qualified (face to face) meeting.
         
         
      8. Voting representative: Each member institution shall designate a voting representative. If the designated voting representative is absent from a meeting, another person employed by the member institution, or its affiliates, may temporarily represent the member. In the event of a lack of clarity as to who has the vote, the chair may refuse to accept a vote until the designated voting representative appoints a substitute in writing.
         
         
6. 1.5. Benchmarks
   
   
   1. 1.5.1. Benchmark Acceptance
      1. Each subcommittee shall document the components that are included in a benchmark.
      2. New or modified benchmark components require a 2/3-majority vote of GWPG eligible voting members to be accepted for publication.
      3. A minimum 3-week review period is required for new or significantly modified benchmark components.
      4. At the end of the review period a vote will be called to approve the proposed changes.
      5. An amendment to a benchmark component during the review period must be unanimously accepted. If not, the review period shall be restarted.
      6. A subcommittee may add requirements for benchmark acceptance as long as they do not conflict with the GWPG Policies and Procedures.
         
         
   2. 1.5.2. Benchmark Release Names and Versions
      1. Benchmarks shall be named according to one of the following two options, at the discretion of each subcommittee. In both options, the minor version of the benchmark must be greater than “00”, and all minor versions of a benchmark must produce comparable scores with other minor versions of the same benchmark,
         1. "SPECtrademark® YYYY v1.NN" where "trademark" is the registered trademark name of the benchmark, "YYYY" is the four-digit year of benchmark release (or the year after), and "NN" is the two-digit minor version of the benchmark,
         2. "SPECtrademark® for APP_NAME APP_VERSION v1.NN" where "trademark" is the registered trademark name of the benchmark, "APP_NAME" is the name of the third-party application to which this benchmark applies; "APP_VERSION" is the version of the third-party application for which this benchmark is valid; and "NN" is the two-digit minor version of the benchmark.
            
            
   3. 1.5.3. Benchmark Release
      1. When a superseded benchmark is retired, the associated run rules and results will be archived along with the benchmark information page.
      2. Each subcommittee shall establish policies and procedures regarding whether results from previous versions of each benchmark may be accepted for publication.
   
   
   
7. 1.6. GWPG Guidelines for Result Submission and Review

   The following sections address the guidelines for submitting benchmark results to the GWPG subcommittees for review and publication on SPEC’s website. Each of the GWPG subcommittees have a set of specific run and reporting rules that the licensee must follow to produce a publishable results.

   Results submitted for publication on the SPEC website must undergo peer review by the respective subcommittee for the benchmark.

   1. 1.6.1. Submission Process
      
      
      1. Privacy The SPEC office will not disclose the submitted results until the submission deadline has passed.
         
         
      2. Rules: The rules for the submission and review cycle to be used are those approved by the respective subcommittee prior to the submission deadline. The approved rules must be posted to the respective subcommittee group’s webpage by the first publication date for the benchmark.
         
         
      3. System declaration: The information supplied in the submission must reflect the system as tested. All fields in the submission’s results file must be supplied with valid entries. The submitter must declare sufficient information in the submission to reproduce the performance claimed.
         
         
   2. 1.6.2. Review Process
      
      
      1. Review Assignment: The subcommittee chair assigns reviewers for submissions. Reviewers must acknowledge assignment by email to the subcommittee email alias. If acknowledgement is not received by email on the second day of the review period, the subcommittee chair may reassign reviewers.
         
         
      2. Conflict of Interest: Members who wish not to review the submission of other specific members due to conflict of interest must submit that list to the subcommittee chair prior to the submission deadline. The subcommittee chair will hold the list in confidence from other members.
         
         
      3. Reviewers: The SPECapc, SPECgpc, and SPECwpc pool of eligible reviewers will be independent of each other. The chair of each subcommittee will send the list of contact information for the submission under review.
         
         
      4. Access: All members will have access to all benchmark submissions once the review period begins.
         
         
      5. Review Period The review period shall be 5 business days, unless a special revision is voted upon by GWPG members.
         
         
      6. Withdrawal: Submissions cannot be withdrawn during the review period without cause and without prior approval of the primary reviewer. A submitter who is granted permission to withdraw a submission must inform the subcommittee by email of the reason for withdrawal. This is to prevent a submitter from withdrawing a submission purely based on the results of other submissions during the same review period.
         
         
      7. Review Questions: If a primary reviewer has a question with a submission, the reviewer must pose the question to the submitter first. If needed, the primary reviewer may also pose questions to the respective subcommittee's officers or GWPG chair for clarification of rules.

         Reviewer questions relating to a submission must do one of the following:

         1. With permission of the primary reviewer, as communicated through the respective subcommittee’s email alias, the submitter may request that his or her submission be rejected on stated technical grounds.
            
            
         2. With permission of the primary reviewer, as communicated through the respective subcommittee’s email alias, a submitter may resubmit a submission to resolve issues found during the review process. The submitter must notify the respective subcommittee’s email list with the date and version of the resubmitted file(s).
            
            
      8. Reproducibility: Results should be reproducible based on the configuration details provided in the submission. The submitter must provide the primary reviewer with additional configuration details as requested during the review period so that the results may be reproduced.
         
         
      9. Review status: By the end of the review period, the primary reviewer of a submission must designate the status of the submission with one of the following: “accepted without comment”, “accepted with comment”, “pending with comment”, or “rejected with comment”. Any comments for rejection of a submission must be received before the end of the review period.
         
         
      10. Pending status: A submission designated “pending with comment” will not go public and will remain pending until the submitter addresses all comments. Once the comments are addressed, the SPEC web master will post to the public website.
          
          
      11. Proposals of non-compliance: If there are open issues for a given submission under review that cannot be resolved to the satisfaction of the subcommittee, the formal process for coming to a resolution is outlined in the SPEC Violations Determination, Penalties and Remedies document.
          
          
      12. Confidentiality: GWPG subcommittee members shall keep all submitted results confidential to the respective subcommittee until those results appear on the public SPEC website. The exception is that members are free to make their own submitted results public at any time, provided the benchmark has been publically announced.
          
          
   3. 1.6.3. Publication Rules
      
      
      1. 1.6.3.1. Fair Use of SPEC Benchmark Results

         Consistency and fairness are guiding principles for SPEC. To help assure that these principles are met, any organization or individual who makes public use of SPEC benchmark results must do so in accordance with SPEC's Fair Use Policy.

      2. 1.6.3.2. SPEC Published Results

         Benchmark results for publication by the SPECgpc, SPECapc, or SPECwpc must adhere to all rules for GWPG and the respective subcommittee.

         A finding of violation is a necessary pre-requisite before SPEC will impose a penalty or require a remedy. The process to determination whether a violation has occurred and appropriate penalty or remedy to be assessed is outlined in the SPEC Violations Determination, Penalties and Remedies document.

      3. 1.6.3.3. Independently Published Results

         Benchmark results for publication outside of the SPEC website, e.g. industry journals, vendor websites, analyst reports, must adhere to all rules for GWPG and the respective subcommittee.

         GWPG and its subcommittees will follow the processes in the SPEC Violations Determination, Penalties and Remedies document when handling proposals of violations for independently published results.

      4. 1.6.3.4. System Availability

         At the time of publication, the system configuration must be generally available within 90 days. Generally available in this context refers to a system configuration which may be purchased in single-unit quantities by the general public via website, phone, or catalog order process.

   4. 1.6.4. Non-member submissions

      The SPEC/GWPG committee will accept submissions from non-members for review and publication on the SPEC public website. Non-member submissions must follow the same rules and procedures as member submissions, except that non-members are not eligible to participate in reviewing results.

      1. Submission Pricing: Non-members will be charged for their submissions according to an approved fee structure. Any change in hardware or software constitutes a new and unique submission.
         
         
      2. Fee Structure: On an annual basis the GWPG will establish the pricing and periods for non-member publication. These will be recorded in the GWPG minutes and published on the GWPG web-site.
         
         
      3. Fee Payment: Following acceptance by the assigned reviewers, a non-member's submission will not be published until the SPEC office has received the submission fee in full. The SPEC office will not deposit funds provided by the non-member submitter until the submission has been accepted by the assigned reviewers.
         
         
      4. Submission Limit: There is a limit of eight non-member submissions per 12-month period. If a company joins GWPG, it will receive a credit for all submissions within the 12-month period before becoming a member.
         
         
      5. Expiration: Each GWPG committee may expire published results from its web pages due to benchmark revision. In this case, a non-member submitter will be given notice by the committee and may resubmit once, the identical configuration for the revised benchmark, for a 12-month period after the initial submission.

Adoption

V1.19 Adopted on 03/03/2020 Updated rule 1.4.3, "Voting", and added rules 1.5.1, "Benchmark Acceptance", 1.5.2, "Benchmark Release Names and Versions", and 1.5.3, "Benchmark Release".

V1.18 Adopted on 08/05/2017 Add rule 1.6.3.4, "System Availability".

V1.17 Adopted on 12/13/2016 reformat, refresh, and clarify rules. Align with SPEC rules on penalties & remedies. Alter non-member submission terms.

V1.16 adopted on 7/30/2016 minor edits, update of meeting notices, correction of non-member publication information

V1.15 adopted on 9/23/2014 removal of pricing and single/multiple supplier rules, addition of SPECwpc, revision of reviewer assignment responsibility

V1.14 adopted on 05/14/2012 new rule IV.2.a, changed rule IV.2.g, and changed wording IV.2.t/u

V1.13 adopted on 04/21/2010 edited rule IV.2.p.iii, and removed duplicate rule, was IV.4.m

V1.12 adopted on 01/27/2010 new rule IV.2.u

V1.11 adopted on 08/13/2009 new/changed rules IV.2.h, IV.4.b and IV.4.m

V1.10 adopted on 09/13/2007 to reflect transition from GPC to GWPG

V1.04 adopted on 10/20/2006

V1.03 adopted on 08/04/2006

V1.02 adopted on 04/27/2006

V1.01 updated on 02/09/2006 to align wording with SPEC policy

V1.00 adopted on 01/25/2006