The purpose of this study was to evaluate the inter-rater reliability of hand diagrams, which are commonly used in research case definitions of carpal tunnel syndrome (CTS). To evaluate the potential of non-random misclassification of cases, we also studied predictors of rater disagreement as a function of personal and work factors, and of hand symptoms not classic for CTS. Participants in a longitudinal study investigating the development of CTS completed repeated self-administered questionnaires. Three experienced clinicians, blind to subjects' work or personal history, independently rated all hand diagrams on an ordinal scale from 0 to 3. Disagreements between ratings were resolved by consensus. Reliability was measured by the weighted kappa statistic. Logistic regression models evaluated predictors of disagreement. Three hundred and thirty-three subjects completed 494 hand diagrams. Eighty-five percent were completed by self-administered questionnaire and 15% by telephone interview. Weighted kappa values representing agreement among the three raters, were 0.83 (95% CI: 0.78, 0.87) for right hand diagrams and 0.88 (95% CI: 0.83, 0.91) for left hand diagrams. Ratings from hand diagrams obtained by telephone interview produced better agreement. Agreement among raters was not affected by subjects' personal or work factors. Disagreement among raters was associated with the presence of hand/wrist symptoms other than classic CTS symptoms. Overall, high levels of agreement were attained by independent raters of hand diagrams. Personal factors did not affect agreement among raters, but presence of non-CTS symptoms seemed to affect results and should be considered in studies focused on diverse populations with heterogeneity of upper extremity symptoms.