Web Tool for Sequential Bayesian Decision Making

    Theodore W. Frick
    Copyright, 2005

    This program demonstrates Bayesian reasoning when attempting to choose one of two mutually exclusive alternatives. (See research study)

    For example, you want to know if your Website or software product is a) working well with your target audience, or b) not working well and needs to be fixed. Or, you may have a large set of usability tasks for your Web site, and you want to know if a given user can either a) do the tasks successfully, or b) not able to do the tasks successfully. Or you may have a student, and you are trying to determine if he or she has a) mastered an educational objective, or b) not mastered it.

    And your job is to find out which alternative is likely to be the case. You will collect some information in order to make a decision, and this program can help you make a decision after each observation, or tell you to collect more information. Before you collect some data, you need to specifiy three sets of things below:

    1. Please enter labels for two alternatives, one of which you need to choose.

    Label for Alternative A (e.g., Mastery of learning objective)

    Label for Alternative B (e.g., Nonmastery of learning objective)

    2. What Is the Expected Success Rate for Each Alternative?

    You need to specify a probability for each of the alternatives. For example, if the site or e-learning product is working well, you would expect most of the users to complete your usability tests successfully, e.g., at least 90%. If the site is not working satisfactorily, the probability of user success would be less, e.g., at most 60%.

    Minimum percent of success if Alternative A is true: (enter a number between 1 and 99)

    Maximum percent of success if Alternative B is true: (enter a number between 1 and one less than if Alternative a) is true)

    3. What Is the Expected Error Rate for a False Conclusion?

    We need to indicate the amount of uncertainty that we can tolerate if we are mistaken about the conclusion. For example, we might be willing to be wrong 5% of the time that the site or e-learning product is working well, when in fact it is not. Similarly, we might be willing to be wrong 5% of the time that the site or product is not working, when in fact it is working satisfactorily.

    Error rate for choosing Alternative B when A is really true = Alpha error = : (number between 1 and 50)

    Error rate for choosing Alternative A when B is really true = Beta error = : (number between 1 and 50)