Brad Porter
Screenwriter
- Joined
- Jun 8, 1999
- Messages
- 1,757
1.0 What is this thread about.
The "Polls Area" draws a strange crowd. The people who gather here are fond of ranking things (movies, actors, women, whatever). There are several methods used to determine the greatest or most popular thing under consideration. The purpose of this thread is to describe the existing methods and to propose new ones, discussing the relative merits of each. There are also other non-ranking thread structures that will be described.
1.1 Who cares?
I'm guessing that there some forum members who:
a) would like to participate in the Polls Area, but can't commit to being here everyday for several weeks or months
b) do participate in the Polls Area, but are getting bored with the current competitive choices
c) do participate, aren't bored, but would enjoy something new
This thread is an opportunity for people to suggest new ideas for running a thread. The subject matter of the threads is not as important to this discussion.
So if you wonder if anybody has ever done something in the Polls Area before, check here for a description of how it may have been done.
1.2 Hasn't anyone done this FAQ thing before?
Well, Scott Weinberg compiled The HTF Tournament FAQ over a year ago. Otherwise, it's been total anarchy around here.
2.0 The existing thread variants
2.1 The non-ranking threads
The Movie Quotes Thread
Members post quotes from movies, either because they like the quotes or to generate the following:
a) A fake conversation using only movie quotes
b) A collection of quotes about a particular subject
The Neverending Story Thread
A member starts a story ("It was a dark and stormy night..."). Other members successively add new paragraphs to the story, taking it in whichever direction they choose.
The Film (or other thing) Association Thread
Basically a series of posts with each one listing a movie that is somehow related (in the poster's mind) to the previously posted movie. They may share a common actor (a la the Kevin Bacon game) or have the same word in the title. There is an ongoing thread which links baseball players by letters in their names.
A Poll (No really!)
The subject matter is wide open. The thread can be started with the choices predetermined or can be open to user input. Some past examples:
a) What do you do for a living?
b) What is your favorite (movie, song, performer, genre)?
c) How much time do you spend at the HTF?
d) How much money have you spent on your home theater investments?
e) How old are you?
f) What's the best looking/sounding/tasting DVD?
g) Have you met someone famous?
h) If you were a tree, what kind of tree would you be? (OK. Just kidding on that one.)
2.2 Competitions
Name That Movie! Screenshot Contests
Usually run by a single member who posts one or more screenshots per day. Members e-mail or send a personal message to the thread creator with their guess on the title of the displayed movie. Points are awarded for correct guesses. The thread ends when the member who started it runs out of screenshots. Some scoring variations have been tried:
a) Award more points if very few people get the shot correct
b) Award points for the worst or funniest guess
c) Deduct points for wrong answers
d) Determine winners based on total points and best percentage
Another variation is to take elements of different screen shots and merge them into a single image with the challenge being to name all of the films used in the image. Creation of these shots takes more time than you might realize (trust me!).
Some cardinal rules for running a screen shot contest:
a) OAR! OAR! OAR!
b) If you modify the image for file size purposes, be careful not to change the coloring or details too much.
c) Don't make the shots too easy, but throw an easier one in once in a while for the less cultured members.
You'll need to have a speedy and reliable server available to support hosting of your images or else you'll get complaints about the "red X".
These are rather popular with the members, but a moderator would have to tell you if the images consume a lot of server bandwidth.
Trivia Contests
I don't know if this one has been done explicitly yet, but some members mix things up in their threads by including trivia questions. Basically, a question is asked and the members post answers. When it's answered correctly, a new question is asked. Points could be awarded for getting the answer correct. It may be better to construct it like the screenshot contests so that there is no benefit based upon when you are on the HTF.
X's favorites vs. Y's favorites
Two members compile a list of their favorite movies. Two movies (One movie from each of these lists) are placed in head to head competition for the member's to vote upon. The member whose list is eliminated first is the loser.
Team Scavenger Hunt
Scott Weinberg ran a scavenger hunt thread using the IMDB message boards (where you can post all kinds of nonsensical threads without moderatus interruptus). The basic principle here is that someone constructs a scavenger hunt using information contained on DVDs. People with small collections could team up with other members to form teams (encouraging participation). The scavenger hunt would be divided up into sections, each of which contains multiple steps.
The first step might be something like this (a fake example):
Go to the third chapter of the film on the Fight Club DVD and get the eighth word spoken by Edward Norton. Go to page 17 of the screenplay on the Taxi Driver DVD and get the 23rd word. Go to 1:12:41 on the There's Something About Mary DVD and get the color of Warren's shirt. Put these words together and generate a movie title (let's pretend it was The Red Shoes). Who directed that movie?
The section would have a time limit during which the competitors PM the thread runner their guess. Points are awarded based upon when the PM was received and if it was correct. This process would continue until the thread creator declares the winner(s).
The biggest barrier to running this thread is setting it up. It's best to work backwards from the answer and find ways to get the competitors to find that answer.
2.3 Tournaments
The thread creator names a topic and gets people to input nominations for competitors in that topic. Any number of members may participate. Sometimes the creator already has a list of things which are in the tournament and do not need to be nominated. Other variations include:
a) A film must be nominated, seconded, thirded, etc. to qualify for the tournament
b) Each participant is restricted in the number of nominations, seconds, thirds, etc. that they can use
c) Participants may be awarded more nominations, seconds, thirds, etc. if they have used all of theirs up already.
Once the nomination limit (2 to the power of x where x = the number of rounds that you want to have; 2 to the power of 6 = 2x2x2x2x2x2 = 64; After the first round you have 32 nominees left; After the second round you have 16 nominees left; third round, 8 nominees; fourth round, 4 nominees, fifth round, 2 nominees, sixth round, champion) has been reached the matchups are set and the voting starts. See below for alternate seeding strategies and alternate voting strategies.
2.4 Drafts
The topic is posted and a call is made for participants. Once the participant limit is reached, the draft order is decided upon and the drafting starts.
Draft order determination
Some people prefer the order that people signed up. Others prefer to randomize the list.
Draft process
Each participant drafts a "thing" when their turn comes up. The general approach is to use a serpentine draft order (1-8, 8-1, 1-8, etc.). Another approach is to use a repeating pattern (1-8, 1-8, 1-8). An approach which hasn't been tried yet but could work well is an auction. Each participant is given a fixed number of points and is allowed to bid on a given "thing". The highest bidder claims the "thing" for their team. Auctions are only useful when the number of "things" is somewhat limited and there are certain things that are clearly more worthy of ownership than others. The goal is to manage your points well, allowing you to develop a good overall team. Given the constraints of having multiple people online at different times, an auction would need to be run with the thread runner not participating. Each participate would PM the thread runner with their bid (which would therefore be blind to all of the other participants). The highest bidder claims the "thing", with no opportunity for a bid off.
One problem with drafting is that sometimes members disappear for awhile. For this reason, draft windows were incorporated. Each drafter was given an exclusive draft window during which no one else could draft. After that time had passed without the drafter making a pick, the draft window for the next drafter started. The original drafter who missed a pick can pick at any time.
During the voting period, one variation is to allow the winning team to raid the losing team for "things". This works as an extension of the draft process and helps to make the later round matchups more interesting.
3.0 Voting and Determining a "Winner"
The standard tournament approach
This evaluation is basically run like the NCAA basketball tournament. For this reason, the number of participants is usually restricted to (2 to the power of X).
Team a
-------- = Team a
Team b
- - - - - -------- = Team a
Team c
-------- = Team d
Team d
This produces a winner in a very quick fashion, but it limits the matchup opportunities to the way in which the seedings were determined. Some people may complain about the seedings being unfair.
---
The Shuffled Matchups
I ran a fight tournament that altered the matchup strategy. Let's say that there were 6 participants. The matchups are determined as follows:
First construct a matrix like this:
Team 1 faces 6, 5, 4, 3, 2
Team 2 faces 5, 4, 3, 2, 1
Team 3 faces 4, 3, 2, 1, 6
Team 4 faces 3, 2, 1, 6, 5
Team 5 faces 2, 1, 6, 5, 4
Team 6 faces 1, 6, 5, 4, 3
See the pattern? Now, since a team can't match up against itself, the matrix needs to be modified. First, elimate these matchups.
Team 1 faces 6, 5, 4, 3, 2
Team 2 faces 5, 4, 3, _, 1
Team 3 faces 4, _, 2, 1, 6
Team 4 faces 3, 2, 1, 6, 5
Team 5 faces 2, 1, 6, _, 4
Team 6 faces 1, _, 5, 4, 3
Then shuffle the numbers as follows. Freeze the matchup numbers for team 1. Now moving down the rows, fill in the blanks with the last team's number, modifying the other matchups as necessary to prevent someone from facing the same team twice.
Team 1 faces 6, 5, 4, 3, 2
Team 2 faces 5, 4, 3, 6, 1
Team 3 faces 4, 6, 2, 1, 5
Team 4 faces 3, 2, 1, 5, 6
Team 5 faces 2, 1, 6, 4, 3
Team 6 faces 1, 3, 5, 2, 4
This format allows for any even number of participants, not just an exponential of 2. The pattern repeats until attrition leaves only one competitor standing. The way that the fight tournament worked involved matching up individual fighters on each team in a head-to-head matchup against the team they were facing. Each team had 6 fighters, meaning that the worst that could happen was that a team would only last six rounds. There were ten teams involved, and the total number of rounds needed to determine a winner was around 15. The matchups didn't generate a single greatest fighter, rather they determined who had the strongest overall team.
What's wrong with it? Things get hairy when teams get eliminated. Suddenly you have an odd number of particpants or you have to reshuffle the matchups because both opponents of two teams are eliminated and each of these teams have just faced each other in the previous round. This really ends up being left to the discretion of the thread runner. For my tournament, I held some extra picks in a pool that I selected from for the odd remaining team to face off against. It would also be possible to grant the most successful team a bye and shuffle the matchups to accommodate it.
The Calculated Rankings
One of the major problems encountered with tournament and draft voting is that people miss the voting window. A useful method to counteract this would be to have a way to leave the voting in all rounds open indefinitely until the victor is crowned. A second issue is that many voters must abstain in a matchup because of an unfamiliarity with one of the "things". Both of these problems are a result of head-to-head matchups, so a numerical ranking method has to be used. This is not particularly useful for evaluating a draft, because there are no head to head matchups comparing one person's "team" with anothers. It is more applicable to the task of ranking a large group of individual "things".
The basic approach is to have all voters give a numerical ranking to each "thing" that they can offer an opinion on. The only rule for this is that everyone agrees to the maximum and minimum values for the ranking. It would also be helpful to give some form of baseline for the rankings, but in a subjective evalution, this isn't usually easy.
Example: Similar to the genre film tournaments, a list of films to be ranked could be generated. Each day one of the films could be discussed (Yes! Actual discussion). At any point during the competition, voters can offer a 0-100 ranking for any of the films in competition. When the voting is declared over, the score acquired by each film is calculated and they are ranked by the score they received. Since not all of the films will have the same number of scores, the values have to be normalized to be fair. If only one person voted for a film and gave it a perfect score (or an awful score), that would heavily influence the rankings towards that one person's opinions. The more people that participate, the more valuable the ranking.
I'm interested in what numerical strategies people would suggest here. The base method would be calculating an average of the scores received, perhaps trimming off the highest and lowest scores first. At any point during the evaluation, new films could be added to the pool, supporting a long running thread which ultimately expresses the cumulative worth of each film as viewed by the conglomeration of the HTF.
4.0 You forgot XXXXXXX (
The "Polls Area" draws a strange crowd. The people who gather here are fond of ranking things (movies, actors, women, whatever). There are several methods used to determine the greatest or most popular thing under consideration. The purpose of this thread is to describe the existing methods and to propose new ones, discussing the relative merits of each. There are also other non-ranking thread structures that will be described.
1.1 Who cares?
I'm guessing that there some forum members who:
a) would like to participate in the Polls Area, but can't commit to being here everyday for several weeks or months
b) do participate in the Polls Area, but are getting bored with the current competitive choices
c) do participate, aren't bored, but would enjoy something new
This thread is an opportunity for people to suggest new ideas for running a thread. The subject matter of the threads is not as important to this discussion.
So if you wonder if anybody has ever done something in the Polls Area before, check here for a description of how it may have been done.
1.2 Hasn't anyone done this FAQ thing before?
Well, Scott Weinberg compiled The HTF Tournament FAQ over a year ago. Otherwise, it's been total anarchy around here.
2.0 The existing thread variants
2.1 The non-ranking threads
The Movie Quotes Thread
Members post quotes from movies, either because they like the quotes or to generate the following:
a) A fake conversation using only movie quotes
b) A collection of quotes about a particular subject
The Neverending Story Thread
A member starts a story ("It was a dark and stormy night..."). Other members successively add new paragraphs to the story, taking it in whichever direction they choose.
The Film (or other thing) Association Thread
Basically a series of posts with each one listing a movie that is somehow related (in the poster's mind) to the previously posted movie. They may share a common actor (a la the Kevin Bacon game) or have the same word in the title. There is an ongoing thread which links baseball players by letters in their names.
A Poll (No really!)
The subject matter is wide open. The thread can be started with the choices predetermined or can be open to user input. Some past examples:
a) What do you do for a living?
b) What is your favorite (movie, song, performer, genre)?
c) How much time do you spend at the HTF?
d) How much money have you spent on your home theater investments?
e) How old are you?
f) What's the best looking/sounding/tasting DVD?
g) Have you met someone famous?
h) If you were a tree, what kind of tree would you be? (OK. Just kidding on that one.)
2.2 Competitions
Name That Movie! Screenshot Contests
Usually run by a single member who posts one or more screenshots per day. Members e-mail or send a personal message to the thread creator with their guess on the title of the displayed movie. Points are awarded for correct guesses. The thread ends when the member who started it runs out of screenshots. Some scoring variations have been tried:
a) Award more points if very few people get the shot correct
b) Award points for the worst or funniest guess
c) Deduct points for wrong answers
d) Determine winners based on total points and best percentage
Another variation is to take elements of different screen shots and merge them into a single image with the challenge being to name all of the films used in the image. Creation of these shots takes more time than you might realize (trust me!).
Some cardinal rules for running a screen shot contest:
a) OAR! OAR! OAR!
b) If you modify the image for file size purposes, be careful not to change the coloring or details too much.
c) Don't make the shots too easy, but throw an easier one in once in a while for the less cultured members.
You'll need to have a speedy and reliable server available to support hosting of your images or else you'll get complaints about the "red X".
These are rather popular with the members, but a moderator would have to tell you if the images consume a lot of server bandwidth.
Trivia Contests
I don't know if this one has been done explicitly yet, but some members mix things up in their threads by including trivia questions. Basically, a question is asked and the members post answers. When it's answered correctly, a new question is asked. Points could be awarded for getting the answer correct. It may be better to construct it like the screenshot contests so that there is no benefit based upon when you are on the HTF.
X's favorites vs. Y's favorites
Two members compile a list of their favorite movies. Two movies (One movie from each of these lists) are placed in head to head competition for the member's to vote upon. The member whose list is eliminated first is the loser.
Team Scavenger Hunt
Scott Weinberg ran a scavenger hunt thread using the IMDB message boards (where you can post all kinds of nonsensical threads without moderatus interruptus). The basic principle here is that someone constructs a scavenger hunt using information contained on DVDs. People with small collections could team up with other members to form teams (encouraging participation). The scavenger hunt would be divided up into sections, each of which contains multiple steps.
The first step might be something like this (a fake example):
Go to the third chapter of the film on the Fight Club DVD and get the eighth word spoken by Edward Norton. Go to page 17 of the screenplay on the Taxi Driver DVD and get the 23rd word. Go to 1:12:41 on the There's Something About Mary DVD and get the color of Warren's shirt. Put these words together and generate a movie title (let's pretend it was The Red Shoes). Who directed that movie?
The section would have a time limit during which the competitors PM the thread runner their guess. Points are awarded based upon when the PM was received and if it was correct. This process would continue until the thread creator declares the winner(s).
The biggest barrier to running this thread is setting it up. It's best to work backwards from the answer and find ways to get the competitors to find that answer.
2.3 Tournaments
The thread creator names a topic and gets people to input nominations for competitors in that topic. Any number of members may participate. Sometimes the creator already has a list of things which are in the tournament and do not need to be nominated. Other variations include:
a) A film must be nominated, seconded, thirded, etc. to qualify for the tournament
b) Each participant is restricted in the number of nominations, seconds, thirds, etc. that they can use
c) Participants may be awarded more nominations, seconds, thirds, etc. if they have used all of theirs up already.
Once the nomination limit (2 to the power of x where x = the number of rounds that you want to have; 2 to the power of 6 = 2x2x2x2x2x2 = 64; After the first round you have 32 nominees left; After the second round you have 16 nominees left; third round, 8 nominees; fourth round, 4 nominees, fifth round, 2 nominees, sixth round, champion) has been reached the matchups are set and the voting starts. See below for alternate seeding strategies and alternate voting strategies.
2.4 Drafts
The topic is posted and a call is made for participants. Once the participant limit is reached, the draft order is decided upon and the drafting starts.
Draft order determination
Some people prefer the order that people signed up. Others prefer to randomize the list.
Draft process
Each participant drafts a "thing" when their turn comes up. The general approach is to use a serpentine draft order (1-8, 8-1, 1-8, etc.). Another approach is to use a repeating pattern (1-8, 1-8, 1-8). An approach which hasn't been tried yet but could work well is an auction. Each participant is given a fixed number of points and is allowed to bid on a given "thing". The highest bidder claims the "thing" for their team. Auctions are only useful when the number of "things" is somewhat limited and there are certain things that are clearly more worthy of ownership than others. The goal is to manage your points well, allowing you to develop a good overall team. Given the constraints of having multiple people online at different times, an auction would need to be run with the thread runner not participating. Each participate would PM the thread runner with their bid (which would therefore be blind to all of the other participants). The highest bidder claims the "thing", with no opportunity for a bid off.
One problem with drafting is that sometimes members disappear for awhile. For this reason, draft windows were incorporated. Each drafter was given an exclusive draft window during which no one else could draft. After that time had passed without the drafter making a pick, the draft window for the next drafter started. The original drafter who missed a pick can pick at any time.
During the voting period, one variation is to allow the winning team to raid the losing team for "things". This works as an extension of the draft process and helps to make the later round matchups more interesting.
3.0 Voting and Determining a "Winner"
The standard tournament approach
This evaluation is basically run like the NCAA basketball tournament. For this reason, the number of participants is usually restricted to (2 to the power of X).
Team a
-------- = Team a
Team b
- - - - - -------- = Team a
Team c
-------- = Team d
Team d
This produces a winner in a very quick fashion, but it limits the matchup opportunities to the way in which the seedings were determined. Some people may complain about the seedings being unfair.
---
The Shuffled Matchups
I ran a fight tournament that altered the matchup strategy. Let's say that there were 6 participants. The matchups are determined as follows:
First construct a matrix like this:
Team 1 faces 6, 5, 4, 3, 2
Team 2 faces 5, 4, 3, 2, 1
Team 3 faces 4, 3, 2, 1, 6
Team 4 faces 3, 2, 1, 6, 5
Team 5 faces 2, 1, 6, 5, 4
Team 6 faces 1, 6, 5, 4, 3
See the pattern? Now, since a team can't match up against itself, the matrix needs to be modified. First, elimate these matchups.
Team 1 faces 6, 5, 4, 3, 2
Team 2 faces 5, 4, 3, _, 1
Team 3 faces 4, _, 2, 1, 6
Team 4 faces 3, 2, 1, 6, 5
Team 5 faces 2, 1, 6, _, 4
Team 6 faces 1, _, 5, 4, 3
Then shuffle the numbers as follows. Freeze the matchup numbers for team 1. Now moving down the rows, fill in the blanks with the last team's number, modifying the other matchups as necessary to prevent someone from facing the same team twice.
Team 1 faces 6, 5, 4, 3, 2
Team 2 faces 5, 4, 3, 6, 1
Team 3 faces 4, 6, 2, 1, 5
Team 4 faces 3, 2, 1, 5, 6
Team 5 faces 2, 1, 6, 4, 3
Team 6 faces 1, 3, 5, 2, 4
This format allows for any even number of participants, not just an exponential of 2. The pattern repeats until attrition leaves only one competitor standing. The way that the fight tournament worked involved matching up individual fighters on each team in a head-to-head matchup against the team they were facing. Each team had 6 fighters, meaning that the worst that could happen was that a team would only last six rounds. There were ten teams involved, and the total number of rounds needed to determine a winner was around 15. The matchups didn't generate a single greatest fighter, rather they determined who had the strongest overall team.
What's wrong with it? Things get hairy when teams get eliminated. Suddenly you have an odd number of particpants or you have to reshuffle the matchups because both opponents of two teams are eliminated and each of these teams have just faced each other in the previous round. This really ends up being left to the discretion of the thread runner. For my tournament, I held some extra picks in a pool that I selected from for the odd remaining team to face off against. It would also be possible to grant the most successful team a bye and shuffle the matchups to accommodate it.
The Calculated Rankings
One of the major problems encountered with tournament and draft voting is that people miss the voting window. A useful method to counteract this would be to have a way to leave the voting in all rounds open indefinitely until the victor is crowned. A second issue is that many voters must abstain in a matchup because of an unfamiliarity with one of the "things". Both of these problems are a result of head-to-head matchups, so a numerical ranking method has to be used. This is not particularly useful for evaluating a draft, because there are no head to head matchups comparing one person's "team" with anothers. It is more applicable to the task of ranking a large group of individual "things".
The basic approach is to have all voters give a numerical ranking to each "thing" that they can offer an opinion on. The only rule for this is that everyone agrees to the maximum and minimum values for the ranking. It would also be helpful to give some form of baseline for the rankings, but in a subjective evalution, this isn't usually easy.
Example: Similar to the genre film tournaments, a list of films to be ranked could be generated. Each day one of the films could be discussed (Yes! Actual discussion). At any point during the competition, voters can offer a 0-100 ranking for any of the films in competition. When the voting is declared over, the score acquired by each film is calculated and they are ranked by the score they received. Since not all of the films will have the same number of scores, the values have to be normalized to be fair. If only one person voted for a film and gave it a perfect score (or an awful score), that would heavily influence the rankings towards that one person's opinions. The more people that participate, the more valuable the ranking.
I'm interested in what numerical strategies people would suggest here. The base method would be calculating an average of the scores received, perhaps trimming off the highest and lowest scores first. At any point during the evaluation, new films could be added to the pool, supporting a long running thread which ultimately expresses the cumulative worth of each film as viewed by the conglomeration of the HTF.
4.0 You forgot XXXXXXX (
