The Graphics and Workstation Performance Group (SPEC/GWPG)
Policies and Procedures Document
Last Updated: 12/13/2016
and Workstation Performance Group (GWPG)
GWPG manages and supervises the development of SPEC’s GWPG
benchmarks. Three subcommittees are under the umbrella of the GWPG:
Performance Characterization subcommittee (henceforth SPECgpcSM)
Performance Characterization subcommittee (henceforth SPECapcSM)
Performance Characterization subcommittee (henceforth SPECwpcSM)
Scope of Rules
- The rules contained in this document shall apply to the GWPG and all
three GWPG subcommittees: SPECgpc, SPECapc, and SPECwpc.
subcommittee shall maintain its own rules document, which shall apply
in conjunction with this document.
a subcommittee’s rule overrides a rule in this document, this
will be explicitly indicated in that subcommittee’s rules
a new subcommittee be approved or an existing one dissolved, this
document shall be updated accordingly.
- In the event of disagreement between this document and SPEC's
or disagreement between this document and decisions by the SPEC Board
of Directors, said Bylaws or decisions shall take precedence over
in the GWPG is open to sustaining member or Associate member. GWPG
subcommittees may also include members who are non-voting supporting
members pay annual dues and accrue all the rights, privileges and
responsibilities that full membership entails. Sustaining memberships
are available to commercial and non-commercial entities that have
direct and/or material interest in graphics or workstation-related
application performance benchmarking. Any eligible entity that wishes
to join GWPG may join provided they pay annual dues as determined by
the Board of Directors.
members may become active participants in any subcommittee of GWPG.
members are eligible to stand for election for the subcommittees,
nominate candidates for offices and the Board of Directors, and to
champion proposed benchmarks.
members have the opportunity to review and comment on all benchmarks
developed by GWPG. This can be done through active participation in
the subcommittee developing the benchmark during any phase of
development from design to testing to release. This participation is
expected but not required.
members are entitled to secure access to benchmark development source
members are entitled to unlimited publication rights, including
benchmark result submissions for publication on the SPEC website.
encourage participation from educational and non-profit institutions
interested in its work, GWPG maintains an Associate category that
enables these organizations to share in the GWPG process, but not to
participate in the general member voting.
status is available to educational and non-profit institutions.
rights and rules of SPEC and GWPG apply to the Associate, unless
specifically stated otherwise.
are entitled to access to development source code.
do not have voting rights.
role of a Supporting Contributor is available by invitation to a
commercial organization, an academic institution, or an individual. A
subcommittee may issue an invitation to an interested party to apply
for this role. The application must be approved by GWPG. Supporting contributors may help develop new benchmarks,
provide expertise to the subcommittee or help provide other support
within the subcommittee.
Structure: Dues paid to the GWPG authorize membership in all GWPG subcommittees.
Dues for the GWPG are determined by the SPEC Board of Directors with
input from the GWPG.
Payment: Members are expected to remain members in good standing by the prompt
remittance of annual dues before the start of the year in question.
Membership privileges will be suspended if dues have not been
received by the SPEC office by March 31st.
Membership privileges will be restored once dues have been paid.
Organization and Voting
On an annual basis, at the first face to face meeting of the year,
GWPG and the subcommittees will elect the following officers from
their eligible voting members: chair, vice-chair, and secretary.
Officer positions are held by individuals.
of Interest: If a conflict of interest affects an officer in pursuance of his or
her duties, and if any defined succession of responsibility fails to
resolve the conflict of interest, the committee may appoint a
committee member to fulfill the officer’s duties for the scope
of the matter in which the conflict of interest exists. For example,
the chair is a party in a re-review or appeal occurrence.
An election will be held to fill a vacancy if an officer steps down,
except for the case where the chair steps down and a vice-chair has
already been elected; in this case, the vice-chair shall become the
chair for the duration of the term, and an election for a new
vice-chair must be held to fill the vacancy.
out an agenda prior to the meetings
interactions with outside organizations such as the press or
and respond on behalf of the group to external question and queries
with the GWPG committee
the submission, review and appeal process, including assigning
members to reviews
the chair’s role when the chair is unavailable, or if the
chair is subject to a conflict of interest.
notes from all meetings
the subcommittee’s rules document
a history of violations
and GWPG subcommittee meetings are conducted in accordance with
general parliamentary procedures under direction of the chair.
chair controls the meeting
participate and aim to reach a consensus
do not interrupt each other or dominate the proceedings
keep an open mind and listen to the opinions of others
subcommittees have three types of meetings: quarterly face to face
meetings, weekly conference call meetings, and special meetings.
to face meetings are the only meetings that are qualified for
attendance. A member loses their voting rights upon missing 2
consecutive qualified meetings.
to face meetings are scheduled at least one month in advance.
Meetings by conference call are scheduled every week unless otherwise
stated due to conflicts that prohibit quorum or holidays. Special
meetings must be scheduled with at least one week notice.
a qualified meeting for which attendance in person is expected, for
example face to face meetings, attending by conference call does not
count as qualified attendance.
Issues may be designated for resolution by casting a vote by eligible
members of the GWPG or subcommittees. Votes may be made by conference
call, in person at a face to face meeting, email, or a combination of
GWPG drives towards consensus among its participants whenever
possible. However to ensure progress, formal member votes will need
to be taken. An active member may vote ‘yes’, ‘no’,
‘abstain’, or ‘pass’ when the vote is first
called. If the member votes ‘pass’, the chair will return
to that member to finalize their vote as either ‘yes’,
‘no’, or ‘abstain’ after the remaining
members have voted. Votes for ‘abstain’ count only toward
establishing quorum. The reasons for voting ‘no’ or
‘abstain’ shall be recorded in the minutes. Such
recording can help with the drive for consensus, as it allows others
to respond to the specific concerns of those who did not vote ‘yes’.
A valid vote requires a quorum. A quorum is met if at least 66% of
eligible voting members respond, i.e. with 5 voting members, 4 must
be present to vote. It is also required that quorum not drop below 3
GWPG does not allow use of proxy voting, that is, the authorization
of an individual who is not an employee of a member institution to
vote on behalf of that member.
GWPG and its subcommittees use the simple majority voting rule,
unless a subcommittee specifies otherwise. Simple majority means the
highest number of votes of ‘yes' or 'no' wins the vote. If
there is a tie, the motion fails.
example of an alternative voting method is "SPEC Consensus
Voting Rule", which means greater than 66% of the votes of ‘yes’
or ‘no’ wins the vote. The advantage of simple majority
is that it may allow progress to be made more quickly; the advantage
of the consensus voting rule is that it may force deeper exploration
of common ground. A proposal to switch voting rules is itself bound
by the rule currently in effect.
table that follows details the number of votes required to pass a
motion if the SPEC Consensus Voting Rule is in effect.
- New members:
New members are eligible for voting rights after the second
consecutive qualified meeting is attended after becoming a member.
A member maintains voting rights by attending at least every other
face to face meeting. For a qualified meeting for which attendance in
person is expected, attending by conference call does not count as
qualified attendance. A member regains voting rights after attending
the second consecutive qualified (face to face) meeting.
Each member institution shall designate a voting representative. If
the designated voting representative is absent from a meeting,
another person employed by the member institution, or its affiliates,
may temporarily represent the member. In the event of a lack of
clarity as to who has the vote, the chair may refuse to accept a vote
until the designated voting representative appoints a substitute in
subcommittee shall document all benchmark related rules in its
respective subcommittee rules document and is responsible for
establishing a regular process for peer review of results.
GWPG Guidelines for Result Submission and Review
following sections address the guidelines for submitting benchmark
results to the GWPG subcommittees for review and publication on
SPEC’s website. Each of the GWPG subcommittees have a set of
specific run and reporting rules that the licensee must follow to
produce a publishable results.
submitted for publication on the SPEC website must undergo peer
review by the respective subcommittee for the benchmark.
- Privacy The SPEC office will not disclose the submitted results until the
submission deadline has passed.
- Rules: The rules for the submission and review cycle to be used are those
approved by the respective subcommittee prior to the submission
deadline. The approved rules must be posted to the respective
subcommittee group’s webpage by the first publication date for
declaration: The information supplied in the submission must reflect the system as
tested. All fields in the submission’s results file must be
supplied with valid entries. The submitter must declare sufficient
information in the submission to reproduce the performance claimed.
Assignment: The subcommittee chair assigns reviewers for submissions. Reviewers
must acknowledge assignment by email to the subcommittee email alias.
If acknowledgement is not received by email on the second day of the
review period, the subcommittee chair may reassign reviewers.
of Interest: Members who wish not to review the submission of other specific
members due to conflict of interest must submit that list to the
subcommittee chair prior to the submission deadline. The subcommittee
chair will hold the list in confidence from other members.
- Reviewers: The SPECapc, SPECgpc, and SPECwpc pool of eligible reviewers will be
independent of each other. The chair of each subcommittee will send
the list of contact information for the submission under review.
- Access: All members will have access to all benchmark submissions once the
review period begins.
Period The review period shall be 5 business days, unless a special revision
is voted upon by GWPG members.
- Withdrawal: Submissions cannot be withdrawn during the review period without
cause and without prior approval of the primary reviewer. A submitter
who is granted permission to withdraw a submission must inform the
subcommittee by email of the reason for withdrawal. This is to
prevent a submitter from withdrawing a submission purely based on the
results of other submissions during the same review period.
- Review Questions: If a primary reviewer has a question with a submission, the reviewer
must pose the question to the submitter first. If needed, the primary
reviewer may also pose questions to the respective subcommittee's
officers or GWPG chair for clarification of rules.
questions relating to a submission must do one of the following:
- With permission of the primary reviewer, as communicated through the
respective subcommittee’s email alias, the submitter may
request that his or her submission be rejected on stated technical
- With permission of the primary reviewer, as communicated through the
respective subcommittee’s email alias, a submitter may resubmit
a submission to resolve issues found during the review process. The
submitter must notify the respective subcommittee’s email list
with the date and version of the resubmitted file(s).
- Reproducibility: Results should be reproducible based on the configuration details
provided in the submission. The submitter must provide the primary
reviewer with additional configuration details as requested during
the review period so that the results may be reproduced.
By the end of the review period, the primary reviewer of a submission
must designate the status of the submission with one of the
following: “accepted without comment”, “accepted
with comment”, “pending with comment”, or “rejected
with comment”. Any comments for rejection of a submission must
be received before the end of the review period.
status: A submission designated “pending with comment” will not
go public and will remain pending until the submitter addresses all
comments. Once the comments are addressed, the SPEC web master will
post to the public website.
of non-compliance: If there are open issues for a given submission under review that
cannot be resolved to the satisfaction of the subcommittee, the
formal process for coming to a resolution is outlined in the SPEC
Violations Determination, Penalties and Remedies document.
- Confidentiality: GWPG subcommittee members shall keep all submitted results
confidential to the respective subcommittee until those results
appear on the public SPEC website. The exception is that members are
free to make their own submitted results public at any time, provided
the benchmark has been publically announced.
Fair Use of SPEC Benchmark Results
and fairness are guiding principles for SPEC. To help assure that
these principles are met, any organization or individual who makes
public use of SPEC benchmark results must do so in accordance with SPEC's Fair Use Policy.
SPEC Published Results
results for publication by the SPECgpc, SPECapc, or SPECwpc must
adhere to all rules for GWPG and the respective subcommittee.
finding of violation is a necessary pre-requisite before SPEC will
impose a penalty or require a remedy. The process to determination
whether a violation has occurred and appropriate penalty or remedy to
be assessed is outlined in the SPEC
Violations Determination, Penalties and Remedies document.
Independently Published Results
results for publication outside of the SPEC website, e.g. industry
journals, vendor websites, analyst reports, must adhere to all rules
for GWPG and the respective subcommittee.
and its subcommittees will follow the processes in the SPEC
Violations Determination, Penalties and Remedies document when
handling proposals of violations for independently published results.
SPEC/GWPG committee will accept submissions from non-members for
review and publication on the SPEC public website. Non-member
submissions must follow the same rules and procedures as member
submissions, except that non-members are not eligible to participate
in reviewing results.
Pricing: Non-members will be charged for their submissions according to an
approved fee structure. Any change in hardware or software
constitutes a new and unique submission.
Structure: On an annual basis the GWPG will establish the pricing and periods
for non-member publication. These will be recorded in the GWPG
minutes and published on the GWPG web-site.
Payment: Following acceptance by the assigned reviewers, a non-member's
submission will not be published until the SPEC office has received
the submission fee in full. The SPEC office will not deposit funds
provided by the non-member submitter until the submission has been
accepted by the assigned reviewers.
- Submission Limit: There is a limit of eight non-member submissions per 12-month period.
If a company joins GWPG, it will receive a credit for all submissions
within the 12-month period before becoming a member.
- Expiration: Each GWPG committee may expire published results from its web pages
due to benchmark revision. In this case, a non-member submitter will
be given notice by the committee and may resubmit once, the identical
configuration for the revised benchmark, for a 12-month period after
the initial submission.
Adopted on 12/13/2016 reformat, refresh, and clarify rules. Align
with SPEC rules on penalties & remedies. Alter non-member
adopted on 7/30/2016 minor edits, update of meeting notices,
correction of non-member publication information
adopted on 9/23/2014 removal of pricing and single/multiple supplier
rules, addition of SPECwpc, revision of reviewer assignment
adopted on 05/14/2012 new rule IV.2.a, changed rule IV.2.g, and
changed wording IV.2.t/u
adopted on 04/21/2010 edited rule IV.2.p.iii, and removed duplicate
rule, was IV.4.m
adopted on 01/27/2010 new rule IV.2.u
adopted on 08/13/2009 new/changed rules IV.2.h, IV.4.b and IV.4.m
adopted on 09/13/2007 to reflect transition from GPC to GWPG
adopted on 10/20/2006
adopted on 08/04/2006
adopted on 04/27/2006
updated on 02/09/2006 to align wording with SPEC policy
adopted on 01/25/2006