How do college students find media online?

According to the Pew Internet & American Life Project, nearly half of Americans use search engines, like Google, daily. According to my browser history, I visit Google 19 times a day, on average! So it’s with that search obsession that in July, I completed my Masters thesis, titled “Media search literacy: The role of access as media technologies converge online.” (Full, 103-page PDF here, or for the gist of it, the Wordle word cloud above sums up all 21,580 words!)

You’re reading a blog, though, so let me condense what I discovered:

A Study of Search Literacy

To be search literate¹ is to possess the skills necessary to effectively find information online via search engines like Google. Media search literacy (MSL) is the ability to find media (like videos and music) online through search engines. The purpose of this study was to gauge college students’ level of MSL and figure out what separates the good searchers from the not-so-good.

Research Procedure

All 102 Illinois State University students (average age: 21) who participated in the study own a computer, have internet access, and say they regularly access media content online. I invited the students to a computer lab where they each completed a short survey. Then they were asked to complete three web-searching tasks, with limited clues, to mimic real-life media searches: (The bold text and hyperlinks below were not provided to the students - only the quoted text.)

1. Find the “Grape Lady” video: “A popular online video features a local newswoman at a winery, crushing grapes with her feet, who falls off a raised platform on live television. You would like to watch this short internet video clip.”
2. Find the Yes song “And You And I”: “You would like to listen to the classic rock song ‘And You And I’ from an album called ‘Close to the Edge.’”
3. Find the first episode of “Angel”: “In 1999, the television drama ‘Angel’ premiered. You would like to watch the first full episode.”²

I recorded video of the students’ computer monitors, so I was able to measure the time they took to complete each task (and if they were successful in under five minutes), the number of pages loaded in their browser (and where they ended up), and search engines they used, the text of their queries, and any “advanced techniques” used.

Highlights from the Results

Natural language is the number-one predictor of poor search effectiveness.

(Natural language is the human mistake of submitting search queries that only other humans could interpret correctly. Most search engines are not sophisticated enough to interpret grammar, questions, or full sentences.)

See table below for other best predictors.

Students use search engines to find media 3 times for every 2 times they use search engines for academic research.

Men were more than twice as likely as women to visit Hulu first to find the TV show, and completed the task more than 2 minutes faster than women on average.

53% of students said they’d received prior search training, mostly in high school, but that experience didn’t improve their search effectiveness.

Most students have a relatively accurate perception of their search abilities and believe they could benefit from search training. Two-thirds said they would attend training.

Students prefer YouTube search over Google for finding media online.

Students from urban high schools spent about 1 minute longer completing each task than students from rural high schools.

The following table shows the behaviors and characteristics of students that correlate to the greatest amount of pageloads and time saved - thus, the best predictors of search effectiveness. For instance, students who submitted natural language search queries spent over a minute longer on each search task, and those who relied on search engines’ suggestions also spent significantly more time.

Best predictors of media search effectiveness

*Comparison of consistent reliance on suggestions versus none.

Students’ Completion of Search Tasks

These next three tables show where and how efficiently these students were able to find the media identified in the search tasks. Most students (95%) could find the “Grape Lady” video, and 3 out of 4 navigated first to YouTube to find it. Less than half of the students were able to complete all three tasks successfully (in under 5 minutes for each). YouTube was also the most popular method for finding the Yes song, and most users found the “Angel” episode on Hulu.² It took the average user 4 minutes 16 seconds to complete all three tasks, and the average user had to open about 19 different webpages to reach all three destinations.

Time and pageloads spent on each search task

*Only includes participants who successfully completed the task (or in the Total row, all tasks).
**Not a sum of above rows, but data on cumulative task performance.
***Percentage represents participants who successfully completed all three tasks.

Top 3 media destinations for successful searchers

Top 3 search engines (SEs) used first and that led to success

*One student completed Task C without using any search engines.

Search Behavior and Advanced Techniques

On average, students submitted about three search queries per task, and used about five keywords per search. Surprisingly, the number of keywords used had no significant effect on search effectiveness. (i.e. Students who used three keywords were as effective as those who used seven.)

Number of searches and average keywords per search

*Not a sum of above rows, but data on cumulative task performance.

I measured the use of several “advanced techniques” in the study, including:

• Boolean operators. (<OR> for expanding a search’s results, <NOT> or <-> for limiting a search’s results.) 22% of students attempted or accidentally used Boolean, but none used it correctly.⁴
• Phrase searching. (Using quotation marks to search keywords only when they appear together.) 30% of students used phrase searching correctly, but it only seemed to help in Task B, searching for the song.

Students who incorrectly used advanced techniques like those above spent an extra 38 seconds per search task (as seen in the best predictors chart above). Paradoxically, students who used advanced search techniques correctly were more likely than their counterparts to also use advanced techniques incorrectly. This suggests training is not comprehensive enough to clear up confusion.

Other Random Factoids I Learned During Research

• In 1829, it took 16 years to manually create an index of all the words in Shakespeare’s plays. Today search engines do this in seconds.
• In one month, Google documented 87 different misspellings of “Britney Spears,” each queried at least 44 times!
• Guess what the most popular search engine query is? Nothing. When users hit the Enter key with an empty search box.

¹I define search literacy as “the ability to identify a searchable information need, access and feed an appropriate search engine an effective, strategic query, utilizing proper keywords and advanced commands (when necessary), evaluate relevance of results, and revise search when necessary.”

²Hulu cycles episode availability, so the premiere episode of “Angel” may not be currently available. During the study, students were able to access the episode on several sites, including Hulu.

³Microsoft’s Live Search became Bing after the study was conducted.

⁴Many students used <-> as a grammatical dash, although most search engines recognize it as a way to remove unwanted results. Some students also used <+> to connect keywords. Although <+> is not incorrect, most search engines don’t require it for including keywords. (All keywords are usually included by default, and Google uses <+> to turn off synonym search.)

Posted in News, Search Engines |
Tagged with Bing, boolean, Fancast, Google, hulu, IMDB, last.fm, Live Search, media, music, research, Rhapsody, search literacy, television, Video, YouTube