EPSY 550 - Rating Scale Design and Analysis (4 Cr)
___________________________________________________________
Email: [email protected]
Phone: 312-996-5630
Office: EPASW 3212
Office Hours: online and phone conference as scheduled
___________________________________________________________
Course Overview
Rating Scale Design and Analysis will prepare students with the skills necessary to develop rating scales designed to measure latent constructs
and questionnaires designed to gather factual information (the emphasis is on rating scales; questionnaires, mailed / telephone / interview surveys,
and sampling are covered extensively by basic introductory texts on these topics as well as by free seminars offered by the SRL -
see http://www.srl.uic.edu/ ), assess reliability and validity for person and item responses, evaluate the functioning of a rating scale, evaluate
model requirements, and analyze and report results using methods based in latent trait theory, specifically Rasch measurement. The course will also
cover Messick's unified view of construct validity as a basis for organizing validation evidence. Topics such as factor/component
analysis, generalizability theory, and reliability and validity from a True Score Theory perspective are covered in EPSY 546 - Educational
Measurement and/or EPSY 583 - Multivariate Statistics. During this course, students have the option to analyze and summarize the results of their
own rating scale data. Examples will be drawn primarily from the fields of education, psychology, and physical rehabilitation.
If you are having technical problems with the course, please click on the "Technical Support" link in the
Blackboard course site to submit a request for assistance or call (312) 996-5948. A st a ff m e m b e r will re sp on d
t o in qu irie s Mon da y - Frida y, 8 a .m . - 8 p.m. CST and Saturday - Sunday, 11 a.m . - 3 p.m . CST.
Prerequisites
EPSY 503 - Essentials of Quantitative Inquiry in Education is required.
EPSY 546 - Educational Measurement or EPSY 561 - Assessment for Measurement Professional or equivalent is recommended .
___________________________________________________________
Course Materials
Required Texts
Download the form here - orderform.pdf
with the books highlighted and ordering directions. Just scan the order form into a pdf and email it to Richard
Sm ith a t in fo @ja m p re s s .o r g. Richard can also take orders over the phone if you are cautious about sending credit
card numbers using e-mail. You can reach Richard at 763-268-2282, 8:00 to 4:30 CST.
• Smith, E.V., & Smith, R.M. (2004). Introduction to Rasch Measurement: Theory, Models, and Applications. Maple Grove, MN: JAM Press.
• Smith, E.V., & Smith, R.M. (2007). Rasch Measurement: Advanced and Specialized Applications. Maple Grove, MN: JAM Press.
• Smith, R.M., & Wind, S.A. (2018). Rasch measurement models: Interpreting WINSTEPS and FACETS output, second edition. Maple Grove, MN: JAM
Press.
Required Technology
You will need regular access to a computer with a connection to the Internet. A dial-up telephone connection will suffice, but high-speed access
via broadband, satellite, or direct subscriber line (DSL) will greatly enhance your online learning experience. This is an online course, and all
course materials and instruction are presented online. A student CANNOT expect to complete the course using only this syllabus as a guide.
Therefore, if you have difficulty accessing the course website, it is imperative that you take action immediately to solve the problem. Do not wait! It
is YOUR responsibility to maintain connectivity throughout the entire length of the course regardless of where you are (this includes vacations,
work-related travel, etc.). Remember that you may access the course from any computer that is capable of searching the Internet, so if you have
difficulty with your home computer, please try a local public computer or a computer at work instead.
You will also need the following software:
• Adobe Acrobat Reader
©
to open PDF files that are used during the course
• You will need to have access to The Microsoft Office suite of software. This includes Word, PowerPoint, and Excel. Work will not be accepted in other
formats for this course
If you do not have SPSS from being enrolled in EPSY 503 (or from work) you will need to obtain the graduate pack of
SPSS Statistical Package. The WebStore (http://webstore.illinois.edu/home/) SPSS for download or direct shipping. If you
want to use a trial version please see https://www.ibm.com/analytics/spss-statistics-
software?lnk=STW_US_STESCH&lnk2=trial_SPSS&pexp=def&psrc=none&mhsrc=ibmsearch_a&mhq=spssThe standard GradPack should be
sufficient for most courses. You may also be able to complete the non-Rasch work of this course with Excel if you
prefer.
___________________________________________________________
Course Goals
In this course, each student will work toward obtaining:
1. An understanding of the basics of questionnaire development and formatting.
2. Knowledge of Winsteps control language and the ability to construct a working Winsteps control file.
3. Knowledge of Rasch terminology.
4. An understanding of the theoretical underpinnings of Rasch measurement.
5. Being able to identify various Rasch models.
6. The ability to select the appropriate Rasch model for different data types.
7. An understanding of the differences between Rasch measurement and IRT.
8. Knowledge of the developmental history of validity theory.
9. An understanding of the unified view of validity theory.
10. An understanding of the development activities for rating scales (mainly self-report).
11. The ability to run Winsteps and interpret output relevant to obtaining validity evidence.
12. The ability to assess item and person fit.
13. The ability to conduct a PCA of residuals.
14. The ability to evaluate local independence.
15. The ability to evaluate the functioning of a rating scale.
16. The ability to determine if data better fits (statistically) the rating scale or partial credit model.
17. The ability to conduct pivot anchoring with the partial credit model.
18. The ability to transform the logit metric for NRT and CRT interpretations.
19. The ability to conduct DIF studies.
20. The ability to evaluate the effectiveness of reverse coding negatively phrased items.
21. An understanding of the multidimensional Rasch model and the advantages of modeling data using a multidimensional approach.
22. The ability to run Conquest for a basic multidimensional Rasch model analysis.
Refer to the course schedule for dates.
___________________________________________________________
Course Requirements
The success of any course is dependent on active and full participation by each class member. It can equally be said that the success of an
individual class member is dependent on the individual's active and full engagement with peers, instructors, and assignments. Each student is
expected to log-in to the course frequently and take full responsibility for engaging with content and monitoring their own progress. Timely
completion of assignments will be the primary evidence of his/her participation in the course. The course is structured with a schedule and weekly
due dates to assist students in appropriately structuring the time they should devote weekly to learning content.
Please note the beginning and ending date for each week in the course schedule and make sure that you complete and
submit required assignments by the ending date and time indicated on the course schedule. Students who wait until the weekends to first check
for the new content postings may encounter difficulties in keeping pace with the course and schedule.
Students are expected to keep current with the readings each week. All of the course readings are either in the primary textbooks (chapters are
listed in the Packet Readings folder) or accessible as a link within the Packet Readings folder. If for any reason you encounter problems with
accessing information you are expected to notify me of the problem as soon as possible and assistance will be provided.
Each student is expected to read and carefully study the reading and module assignments. It is the responsibility of each student to monitor
his/her progress in meeting learning objectives and understanding the content and application of knowledge acquired each week via completion of
the readings, modules, and homework. If a student determines that he/she is not mastering the content covered each week, it is his/her
responsibility to contact the instructor to seek assistance. Such assistance may include asking additional questions, requesting clarification on a
topic, requesting a phone conference, inquiring about additional instructional resources, etc.
Your final grade will be calculated by weighting your performance on the following course requirements:
Homework: There are a total of 11 homework assignments. The first begins in week 5 (there is a lot of reading in
weeks 1 through 4 so no homework will be assigned during those periods). Each homework assignment will ask the
student to apply the concepts taught that week and submit by the end date (see the schedule) their results
via BlackBoard. The optional homework will allow students to apply the same concepts to their own data for additional
practice. Feedback will be provided on both the required and optional homework; however only the required
homework will be graded.
Final Exams: You will have two graded exams in this course. Both come at the end of the course. The Written examination
will be a traditional examination consisting of multiple-choice, true-false, short answer, and computer output interpretations. For the Applications
examination, I will provide you with a raw data file and a code book. You will need to develop the Winsteps control file, run the specified analysis,
and then respond to a series of questions concerning the analyses. Conquest will not be part of the Applications examination.
The final grade will be determined by students' performance on the two examinations and 11 homework assignments. The weight assigned to
each of these components for computation of the final grade is as follows:
Exams 45% (20% for the Written and 25% for the Applications examination)
Homework 55% (5% for each homework).
Note 1: If you want to do the additional optional work but do not have access to data here are a few websites that
a llo w a cce ss t o d a t a s u ita b le fo r th is co u rs e :
• http://www.icpsr.umich.edu/icpsrweb/ICPSR/
• https://nces.ed.gov/surveys/pisa/datafiles.asp
Note 2: After this courses is done, if you want to put all these practice analyses into a manuscript suitable for
publication or presentation I have provided a Guidelines for Manuscripts folder within the Packet Readings. The
Journal of Applied Measurement guidelines are provided as well as 4 validity papers to use as examples of what is
expected in a Rasch oriented publication. After you complete a polished manuscripts I would be glad to review it and
provide feedback. Please note that proper IRB approval is needed for using any analyses of your own data that are to be used for
anything outside of practicing the skills taught in this course.
___________________________________________________________
Course Grades
A
90% -
100%
Excellent
The student's work demonstrates excellent
grasp of almost all the learning outcomes
associated with the course.
B
79% -
89%
Good
The student's work demonstrates mastery of
the majority of learning outcomes associated
with the course.
C
68% -
78%
Average
The student's work demonstrates mastery of
approximately two-thirds of the learning
outcomes associated with the course.
D
57% -
67%
Poor
The student's work demonstrates mastery of
fewer than approximately half of the learning
outcomes associated with the course.
F
56% -
Lower
Failure
The student's work does not sufficiently
demonstrate that he or she has adequately
grasped most of the learning outcomes
associated with the course.
___________________________________________________________
Course Policies
Incompletes
Incompletes will be considered for students with extenuating circumstances. Poor performance on the exams or assignments will not be
considered in a request for an incomplete.
Late Assignments
Late assignments will not be accepted so please pay careful attention to the end date for each assignment (see the schedule). It would behoove
you to contact me AT LEAST ONE DAY BEFORE the DUE DATE if you know something will be late to receive an approved extension. The
bottom line here is keep me informed!
Missed Assignments
Students not submitting a homework or exam will receive a grade of zero. If you are ill or have a serious problem that prevents you from turning in
homework or exam on time, please refer to the late assignment policy above.
Academic Integrity
UIC is committed to upholding academic integrity among all of its students, faculty, staff, and administration. The students and instructor of this
course share this responsibility by not engaging in behaviors that constitute academic dishonesty and misconduct. Examples of such misconduct
include cheating, taking an examination by proxy, plagiarizing, and submitting another person's work as your own. To detect instances of
plagiarism and similar infractions, your work in the course may be scanned with plagiarism detection tools (such as SafeAssign). When evidence of
plagiarism or other academic misconduct occurs, the instructor and University will take action in accordance with the Student Disciplinary Policy.
Students who violate the policies governing academic dishonesty are subject to penalties such as receiving a failing grade for the course and
dismissal from the University. You should review the policy and frequently asked questions from this Student Disciplinary Policy 17-18
_FINAL_.pdf
.
ADA Policy
UIC strives to ensure the accessibility of programs, classes, and services to students with documented disabilities. Reasonable accommodations
can be arranged for students with various types of disabilities, such as documented learning disabilities, vision or hearing impairments, and
emotional or physical disabilities. If you need accommodations for this course, be sure to register with the Office of Disability Services [1190 SSB,
312-413-2183 (voice), 312-413-0123 (TTY only)].
EPSY 550 - Rating Scale Design and Analysis (4 Cr)
___________________________________________________________
Week
Begin
Date
End
Date
Readings Modules/Packets Assignments Due (At 11:59 CST on End Date)
1
Wednesday
1/13
Tuesday
1/19
Course
Syllabus
Course
Schedule
Course
Resources
Packet 1
Readings
Online Learning in MESA
EPSY 504 Course
Orientation
Packet 1:
Measurement Process
No assignment due but be sure to:
Obtain ALL required course textbooks and software
Post and reply in the Ice Breaker Discussion
Visit the Water Cooler Discussion
2 1/20 1/26
Packet 2
Readings
Packet 2: (Part I and II):
Instrument Development
None
3 1/27 2/2
Packet 3
Readings
Packet 3 (Part I):
Rasch Measurement
None
4 2/3 2/9
Packet 3
Readings
Packet 3 (Part II):
Rasch Measurement
None
5 2/10 2/16
Packet 4
Readings
Packet 4:
Measurement and Validity
Create a construct map for Liking for Science data
Optional: Create a construct map for your own data
6 2/17 2/23
Packet 5
Readings
Packet 5 (Part I and IIA):
Winsteps
Complete the ATD control file
Optional: Complete a control file for your data
7 2/24 3/2
Packet 5
Readings
Packet 5 (Part IIB and IIC):
Winsteps
Run and answer questions for the ATD control file (covers Parts IIA and IIB)
Optional: Run and answer the same questions for your data
8 3/3 3/9
Packet 5
Readings
None for
Packet 6
Packet 5 (Part IID):
Winsteps
Packet 6:
Model Selection - Rating
Scale or Partial Credit
Further analysis of the Inattentive ADHD items
Decide on the RSM or PCM for the Liking for Science data
Optional: Decide on the RSM or PCM for your data, if applicable
9 3/10 3/16
Packet 7
Readings
Packet 7:
Rating Scale Functioning
Report on rating scale functioning for the Classroom Technology
Questionnaire
Optional: Report on the rating scale functioning with your data, if applicable
10 3/17 3/23
Packet 8
Readings
Packet 9
Readings
Packet 8:
Pivot Anchoring with
the Partial Credit
Model
Packet 9:
Metric Development
Pivot anchoring with ATD data
Metric development with ATD data
Optional: Conduct pivot anchoring, if applicable, and metric transformation
with your data
11 3/24 3/30
Spring
Break
To be consistent with our Wednesday to Tuesday schedule I
have specified 3/24 to 3/30.
12 3/31 4/6
Packet 10
Readings
Packet 10 (Part I and II):
DIF
DIF with ATD data
Optional: DIF with your data
13 4/7 4/13
Packet 11
Readings
Packet 11:
Multidimensional Models
Estimating a 3 dimensional model with ADHD data
Optional: Run theoretically competing models with your own data and
compare relative model fit
14 4/14 4/20
Packet 12
Readings
Packet 12:
Reverse Coding
Evaluation of reverse coding with ATD data
Optional: Evaluation of reverse coding with your data, if applicable
Take the practice mock exam
Respond to the course evaluation request
15 4/21 4/27
Written and Application
Examinations
The exams will be open from 4/20 11:59 PM CST to 4/27 11:59 PM
CST. You must access, complete, and submit both exams (Written and
Application) during this time period.
Once you open the Written exam you will have 4 hours to complete and
submit it in one sitting. The Application exam has no time limit except for the
4/27 submission deadline.
___________________________________________________________
