Reliability and Validity

Reliability

The National Spanish Assessment recognizes that test reliability is defined as the degree to which the test gives consistent results each time it is given.

 

In other words, reliability answers the following questions:

  1. Can I depend on the test to measure the same outcomes consistently?
  2. Given all the other variables being the same, will the test produce the same results again?

 

 Each question on NSA has been tested at specific levels similar to NSE and correlated to ACTFL proficiency levels.  Please reference NSE validity here for additional information.

 

 

Validity

 NSA recognizes three types of evidence which can be collected to ensure test validity:

1.      criterion-related evidence
Evidence demonstrating the systematic relationship of test scores on a predictor test to a predicted criterion variable.
2.      construct-related evidence
Empirical evidence that (1) supports the posited existence of a hypothetical construct and (2) indicates an assessment device does, in fact, measure that construct.
3.      content-related evidence
Evidence indicating that an assessment instrument suitably reflects the content domain it is supposed to represent

 

Choosing the type of evidence to collect is based on the type of test.  Because criterion-related evidence of validity is concerned with aptitude tests, the NSA does not use this type of evidence in assessing validity of items on the National Spanish Examination.  Similarly, the NSA does not collect construct-related evidence, because there are no hypothetical constructs assessed on the NSA. 

 

Content-related evidence is the most important in assessing validity for the National Spanish Assessment, because the achievement section of the National Spanish Assessment is designed from specific learner outcomes based on content standards for specifications for vocabulary and grammar.  Furthermore, content-related evidence of validity is ensured through (1) development care and (2) external reviews.  Furthermore, items in the proficiency section (Interpretive Reading and Interpretive Listening) are validated through a high number of samples which are placed accordingly into the applicable level, similar to National Spanish Exams. More information can be found here.