Item Development and Analysis Worksheet 2

Table of Contents

Item Development and Analysis

PART 1: Writing Multiple Choice Test Items

Save Time On Research and Writing
Hire a Pro to Write You a 100% Plagiarism-Free Paper.
Get My Paper
Item Development and Analysis

Develop one multiple choice question that covers content from each of the four chapters listed below.  When writing your sample questions, please keep in mind the specifications regarding item construction discussed in the textbook.  Also, remember the importance of carefully crafted distractor options.  Finally, please limit the number of response options to 4 (1 correct response and 3 distractors), and avoid the options of “all of the above,” none of the above,” or the like.  Be sure to indicate which of the response options is the correct one.

Chapter 3 Multiple Choice Question (2.5 points)

An estimate of the relaibility of a speed test is a measure of ?

A) the consistancy of flood

Save Time On Research and Writing
Hire a Pro to Write You a 100% Plagiarism-Free Paper.
Get My Paper

B) the consistancy of response

C) the consistancy of the response speed

D) the consistancy of the response of intensity

Chapter 4 Multiple Choice Question (2.5 points)

Chapter 5 Multiple Choice Question (2.5 points)

Chapter 6 Multiple Choice Question (2.5 points)

PART 2: Item Analysis: Item Difficulty Index(Cohen et al., 2013, pg. 263)

A test is only as good as its questions!  When researchers, test constructors, and educators create items for ability or achievement tests, we have a responsibility to evaluate the items and make sure that they are useful and high-quality.  The process that we use to evaluate test items is known as Item Analysis.  When bad items are identified and eliminated from a test, that increases the efficiency, reliability and validity of the entire test!    One way that we can distinguish among good and bad items is with the Item Difficulty Index

Part 2A: Calculating Item Difficulty

Item Development and Analysis

Using the data below, calculate the Item Difficulty Index for the first 6 items onQuiz 1 from a recent section of PSYC101.  For each item, “1” means the item was answered correctly and “0” means it was answered incorrectly. Type your answers in the spaces provided at the bottom of the table. (1 pt. each)

PSYC101 Quiz 1 Item Distribution and Total Scores
ExamineeItem 1Item 2Item 3Item 4Item 5Item 6Total Score
Andre11111116
Allison1111007
Heather11110010
Corey11011117
Christina0011013
Jeffrey01110011
Shawn11110114
Dana00110110
Megan11110113
David01110112
Isabel0001004
Lance1111009
Aliyah11110115
Blaire01110112
Gabriel0011006
Item
Difficulty
53.33373.33386.66710013.33360 

Part 2B: Calculating Optimal Item Difficulty (.5 pt. each)

Item Development and Analysis

1.      For a test item with two response options (e.g., true/false), what is the probability of selecting the correct answer by chance?

                  %

2.      Calculate the optimum level of difficulty for a test questions with two response options.

                  %

3.      For a test item with three response options, what is the probability of selecting the correct answer by chance?

                  %

4.      Calculate the optimum level of difficulty for a test questions with three response options.

                  %

5.      For a test item with four response options, what is the probability of selecting the correct answer by chance?

                  %

6.      Calculate the optimum level of difficulty for a test questions with four response options.

                  %

7.      For a test item with five response options, what is the probability of selecting the correct answer by chance?

                  %

8.      Calculate the optimum level of difficulty for a test questions with five response options.

                  %

PART 3: Item Analysis: Item Discrimination Index(Cohen et al., 2013, pg. 265–266)

Another way that test creators can distinguish between good and bad items is with an analysis called the Discrimination Index.  The discrimination index measures how well an individual test item distinguishes between high scorers and low scores on the test.  An item is considered to be “good” if most of the high scorers get it right, and most of the low scorers get it wrong.

Interpreting the Discrimination Index (d)

·         The discrimination index can range from -1.0 to 1.0.

·         The closer d is to 1.0, the better the item discriminates between high and low scorers

·         The closer d is to 0, the more poorly the item discriminates between high and low scorers.

·         An item with a negative discrimination index is considered a “negative discriminator” because more low scorers get the item correct than high scorers.

·         A discrimination index of 1.0 means all the high scorers got the item correct and all of the low scorers got it incorrect.

·         A discrimination index of -1.0 means all of the low scorers got the item correct and all of the high scorers got it incorrect.

·         Items with d’s close to 0 or with negative d’s ought to be eliminated from the test!

Calculating the Item Discrimination Index (d)

Calculate the item discrimination index (d) for the 7 hypothetical test items presented below.  Type your answers in the spaces provided at the right of the table (1 pt. each).

Item #ULnd
Item 1211725 
Item 223725 
Item 325025 
Item 432425 
Item 522325 
Item 602525 
Item 719625 

Based on your calculations above, answer the following questions (1 pt. each).

1.      Which item discriminates the best?                                             

2.      Which item discriminates most poorly?                                                  

3.      Based on your analysis, identify which two items would you choose to eliminate from this test and explain why you would eliminate each.

Part 4: Item Characteristic Curves (Cohen et al., pg. 268–270)

Another method that test creators can use to assess the usefulness of test items is with Item Characteristic Curves.  Item characteristic curves provide a graphical depiction of examinees’ performance on individual test items.  As indicated in the figure below, Total Test Score is plotted on the x-axis of the curve, while proportion of examinees who got the item correct is plotted on the y-axis

Using the figure above, provide a written description of how test items A–D discriminate among examinees at various levels of performance.  In your responses, discuss why each item would be considered a “good” or a “bad” item.  EXAMPLE: “This item discriminates well among high scores, but doesn’t discriminate well among low scorers.  So this item would be considered a good item because it discriminates at the highest levels of performance.” (2 pt. each) 

Item A:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

Item B:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

Item C:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

Item D:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

Item E:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   

Part 5: Qualitative Item Analysis (Cohen et al., pg. 272–274)

Qualitative item analysis refers to a set of non-statistical procedures used to gather information about the usefulness of test items.  These analyses typically involve interviews, panel discussions, questionnaires and other forms of verbal exchange with test-takers to explore how individual test items work.

As an online student, you have a very different test-taking experience than residential students.  Based on your readings from Chapter 8, identify 4 topics related to online test taking, and create 4 qualitative questions that you could ask online test-takers to gain an understanding of their experiences with test-taking.  Also, as students at a Christian institution of higher education, course assignments/assessments are supposed to give students an opportunity to integrate course content with their Christian worldview.  Given the topic of faith and learning, create one qualitative question that you could ask test-takers.

Qualitative Item Analysis
Topic (1 pt. each)Sample Question for Test-Takers (1 pt. each)
    
    
    
    
    

Assignment Scoring

Part 1 Subtotal:                      

Part 2 Subtotal:                      

Part 3 Subtotal:                      

Part 4 Subtotal:                      

Part 5 Subtotal:                      

ITEM DEVELOPMENT AND ANALYSIS WORKSHEET

Student Name: Section: PSYC421-

PART 1: Writing Multiple Choice Test Items

Develop one multiple choice question that covers content from each of the four chapters listed below.

When writing your sample questions, please keep in mind the specifications regarding item

construction discussed in the textbook. Also, remember the importance of carefully crafted distractor

options. Finally, please limit the number of response options to 4 (1 correct response and 3

distractors), and avoid the options of “all of the above,” none of the above,” or the like. Be sure to

indicate which of the response options is the correct one.

Chapter 3 Multiple Choice Question (2.5 points)

An estimate of the relaibility of a speed test is a measure of ?

A) the consistancy of flood

B) the consistancy of response

C) the consistancy of the response speed

D) the consistancy of the response of intensity

Chapter 4 Multiple Choice Question (2.5 points)

Chapter 5 Multiple Choice Question (2.5 points)

Chapter 6 Multiple Choice Question (2.5 points)

PSYC 421

Page 2 of 6

PART 2: Item Analysis: Item Difficulty Index (Cohen et al., 2013, pg. 263)

A test is only as good as its questions! When researchers, test constructors, and educators create items

for ability or achievement tests, we have a responsibility to evaluate the items and make sure that they

are useful and high-quality. The process that we use to evaluate test items is known as Item Analysis.

When bad items are identified and eliminated from a test, that increases the efficiency, reliability and

validity of the entire test! One way that we can distinguish among good and bad items is with the

Item Difficulty Index.

Part 2A: Calculating Item Difficulty

Using the data below, calculate the Item Difficulty Index for the first 6 items on Quiz 1 from a recent

section of PSYC101. For each item, “1” means the item was answered correctly and “0” means it was

answered incorrectly. Type your answers in the spaces provided at the bottom of the table. (1 pt. each)

PSYC101 Quiz 1 Item Distribution and Total Scores

Examinee Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Total Score

Andre 1 1 1 1 1 1 16

Allison 1 1 1 1 0 0 7

Heather 1 1 1 1 0 0 10

Corey 1 1 0 1 1 1 17

Christina 0 0 1 1 0 1 3

Jeffrey 0 1 1 1 0 0 11

Shawn 1 1 1 1 0 1 14

Dana 0 0 1 1 0 1 10

Megan 1 1 1 1 0 1 13

David 0 1 1 1 0 1 12

Isabel 0 0 0 1 0 0 4

Lance 1 1 1 1 0 0 9

Aliyah 1 1 1 1 0 1 15

Blaire 0 1 1 1 0 1 12

Gabriel 0 0 1 1 0 0 6

Item

Difficulty

53.333 73.333 86.667 100 13.333 60

PSYC 421

Page 3 of 6

Part 2B: Calculating Optimal Item Difficulty (.5 pt. each)

1. For a test item with two response options (e.g., true/false), what is the probability of selecting the correct answer by chance?

%

2. Calculate the optimum level of difficulty for a test questions with two response options. %

3. For a test item with three response options, what is the probability of selecting the correct answer by chance?

%

4. Calculate the optimum level of difficulty for a test questions with three response options. %

5. For a test item with four response options, what is the probability of selecting the correct answer by chance?

%

6. Calculate the optimum level of difficulty for a test questions with four response options. %

7. For a test item with five response options, what is the probability of selecting the correct answer by chance?

%

8. Calculate the optimum level of difficulty for a test questions with five response options. %

PART 3: Item Analysis: Item Discrimination Index (Cohen et al., 2013, pg. 265–266)

Another way that test creators can distinguish between good and bad items is with an analysis called

the Discrimination Index. The discrimination index measures how well an individual test item

distinguishes between high scorers and low scores on the test. An item is considered to be “good” if

most of the high scorers get it right, and most of the low scorers get it wrong.

Interpreting the Discrimination Index (d)

 The discrimination index can range from -1.0 to 1.0.

 The closer d is to 1.0, the better the item discriminates between high and low scorers

 The closer d is to 0, the more poorly the item discriminates between high and low scorers.

 An item with a negative discrimination index is considered a “negative discriminator” because

more low scorers get the item correct than high scorers.

 A discrimination index of 1.0 means all the high scorers got the item correct and all of the low

scorers got it incorrect.

 A discrimination index of -1.0 means all of the low scorers got the item correct and all of the

high scorers got it incorrect.

PSYC 421

Page 4 of 6

 Items with d’s close to 0 or with negative d’s ought to be eliminated from the test!

Calculating the Item Discrimination Index (d)

Calculate the item discrimination index (d) for the 7 hypothetical test items presented below. Type

your answers in the spaces provided at the right of the table (1 pt. each).

Item # U L n d

Item 1 21 17 25

Item 2 23 7 25

Item 3 25 0 25

Item 4 3 24 25

Item 5 22 3 25

Item 6 0 25 25

Item 7 19 6 25

Based on your calculations above, answer the following questions (1 pt. each).

1. Which item discriminates the best?

2. Which item discriminates most poorly?

3. Based on your analysis, identify which two items would you choose to eliminate from this test and explain why you would eliminate each.

Part 4: Item Characteristic Curves (Cohen et al., pg. 268–270)

Another method that test creators can use to assess the usefulness of test items is with Item

Characteristic Curves. Item characteristic curves provide a graphical depiction of examinees’

performance on individual test items. As indicated in the figure below, Total Test Score is plotted on

the x-axis of the curve, while proportion of examinees who got the item correct is plotted on the y-axis

PSYC 421

Page 5 of 6

Using the figure above, provide a written description of how test items A–D discriminate among

examinees at various levels of performance. In your responses, discuss why each item would be

considered a “good” or a “bad” item. EXAMPLE: “This item discriminates well among high scores,

but doesn’t discriminate well among low scorers. So this item would be considered a good item

because it discriminates at the highest levels of performance.” (2 pt. each)

Item A:

Item B:

Item C:

Item D:

Item E:

Part 5: Qualitative Item Analysis (Cohen et al., pg. 272–274)

Qualitative item analysis refers to a set of non-statistical procedures used to gather information about

the usefulness of test items. These analyses typically involve interviews, panel discussions,

questionnaires and other forms of verbal exchange with test-takers to explore how individual test items

work.

As an online student, you have a very different test-taking experience than residential students. Based

on your readings from Chapter 8, identify 4 topics related to online test taking, and create 4 qualitative

questions that you could ask online test-takers to gain an understanding of their experiences with test-

taking. Also, as students at a Christian institution of higher education, course assignments/assessments

are supposed to give students an opportunity to integrate course content with their Christian

worldview. Given the topic of faith and learning, create one qualitative question that you could ask

test-takers.

PSYC 421

Page 6 of 6

Qualitative Item Analysis

Topic (1 pt. each) Sample Question for Test-Takers (1 pt. each)

Assignment Scoring

Part 1 Subtotal:

Part 2 Subtotal:

Part 3 Subtotal:

Part 4 Subtotal:

Part 5 Subtotal:

Calculate the price
Make an order in advance and get the best price
Pages (550 words)
$0.00
*Price with a welcome 15% discount applied.
Pro tip: If you want to save more money and pay the lowest price, you need to set a more extended deadline.
We know how difficult it is to be a student these days. That's why our prices are one of the most affordable on the market, and there are no hidden fees.

Instead, we offer bonuses, discounts, and free services to make your experience outstanding.
How it works
Receive a 100% original paper that will pass Turnitin from a top essay writing service
step 1
Upload your instructions
Fill out the order form and provide paper details. You can even attach screenshots or add additional instructions later. If something is not clear or missing, the writer will contact you for clarification.
Pro service tips
How to get the most out of your experience with Scholary Essays
One writer throughout the entire course
If you like the writer, you can hire them again. Just copy & paste their ID on the order form ("Preferred Writer's ID" field). This way, your vocabulary will be uniform, and the writer will be aware of your needs.
The same paper from different writers
You can order essay or any other work from two different writers to choose the best one or give another version to a friend. This can be done through the add-on "Same paper from another writer."
Copy of sources used by the writer
Our college essay writers work with ScienceDirect and other databases. They can send you articles or materials used in PDF or through screenshots. Just tick the "Copy of sources" field on the order form.
Testimonials
See why 20k+ students have chosen us as their sole writing assistance provider
Check out the latest reviews and opinions submitted by real customers worldwide and make an informed decision.
Sociology
Thanks to the entire team!
Customer 452919, December 8th, 2021
Other
Should have been around social worker in the UK but I can adapt it.
Customer 453101, August 5th, 2022
Sociology
I have never experienced receiving a paper past the due date and time. That is the only thing that displeases. I don't have time o Overall, your team does a great job.
Customer 452919, November 18th, 2021
English 101
Great work!!
Customer 452989, November 21st, 2021
Psychology
Paper was received before time which I was quite please with.
Customer 453101, June 18th, 2022
Business Studies
The writer delivered as on time with quality work. But writteres communication could of been better and I would have been more content with choosing my own writer as per my assignment.
Customer 452679, March 24th, 2021
Humanities
Best Serivces
Customer 452995, January 24th, 2022
Logistics
The presentation slides were not narrated as asked per the instructions.
Customer 452623, September 28th, 2021
Nursing
n/a
Customer 452725, April 23rd, 2021
Humanities
Always excellent service, appreciate all the hard work
Customer 452995, January 20th, 2022
Business Studies
Thank you sincerely great paper and content
Customer 452793, June 22nd, 2021
Professional Writing Engl-3311
Thank you
Customer 452919, March 2nd, 2023
11,595
Customer reviews in total
96%
Current satisfaction rate
3 pages
Average paper length
37%
Customers referred by a friend
OUR GIFT TO YOU
15% OFF your first order
Use a coupon FIRST15 and enjoy expert help with any task at the most affordable price.
Claim my 15% OFF Order in Chat
Live Chat+1(978) 822-0999EmailWhatsApp

Order your essay today and save 30% with the discount code ESSAYHELP