Professional Practice Skills

PPS-25 Self-Directed Learning

Evaluating Resources

Pre-class assignment

1. Read sections What is It?, Why Do It?, New Concepts, How to Do It, and Learning Objectives
2. Establish your Baseline on this skill on the Feedback Form.
3. Be able to describe

What is It?

Evaluating resources is a method to quantify a written resource’s “quality”.

New Concepts

Authoritative, Quantitative, Unbiased, Thorough

Why Do It?

Not all sources are created equal. In particular, the Web has allowed easy publication and distribution of all information without regard for quality. Novices sometimes have difficulty distinguishing between resources. This method is intended to help novices learn to recognize quality.

How to Do It

When you find sources, you will need a way to decide how “good” they are. The rating scheme below attempts to numerically rate sources with respect to “quality”. In this scheme a quality source in the engineering world is

· Authoritative

· Quantitative

· Unbiased

· Thorough

We are going to look at each of these criteria in turn and try to quantify the quality of our source.

Authoritative

If we want to know the density of Osmium at room temperature, the CRC Handbook of Chemistry and Physics is more authoritative than Uncle Jake’s web site. Authority in science and engineering comes from a history of accuracy and a reputation for knowledge in the field. This is the result of one of the most powerful controls in science, peer review. Consequently, handbooks published by professional societies and articles in refereed journals command considerable respect. Unfortunately, most practicing engineers must get the bulk of their information from trade journals, vendors, and web sites. Therefore, we need some way of rating their authority.

Score	Authoritative
1	Authorship of information is unknown/unclear. Information is of unknown source.
2	Author is known but lacks recognized standing (student, sales people, experts outside their field). Author is not the origin of info and little or no reference to better authority exists. (Most small company web sites fall into this category.)
3	Author has reasonable recognition. Information is referenced to secondary sources.(Trade journal articles are often like this.)
4	Author is recognized and reputable. Information is probably good, but is second hand and is inadequately referenced to a primary source. (Many textbooks fall into this category)
5	Author is recognized and reputable. (Can include college faculty, reputable corporations as well as professional societies) Information was created by the author (GE test data on Lexan) or is adequately referenced to original source. (Refereed journals are in this category)

Quantitative

To be most useful to a designer, engineering information should be quantitative. A vendor can say their new plastic is “better” but it is more useful to know that it is 30% stronger than nylon. Even better is knowing that the material has ultimate tensile strength of 12,000 psi when tested according to ASTM D638. The best information is reported as quantitative values referenced to known engineering standards.

Score	Quantitative
1	Information is in the form of adjectives (better, stronger, lighter)
2	Adjectives have vague reference (lighter than Kevlar, stronger than steel)
3	Information is well defined in a relative sense (material is 30% stronger than cold rolled 1020 steel)
4	Information is numeric without reference to standards (UTS=12,000 psi)
5	Information is numeric and referenced to specific standards. (UTS=12,000 psi per ASTM D638)

Unbiased

If you have ever sold or bought a used car, you know that sales people (including yourself) are not always forthcoming with all the details, especially the disadvantages. Consequently, no commercial source can be considered unbiased. They may be authoritative, quantitative, and thorough, but they will not be unbiased.

Score	Unbiased
1	Commercial web sites, press releases, and most short articles in trade journals such as Machine Design are simply some form of advertising.
2	The work of only one person or company, rather than information that has be independently verified by other individuals or groups.
3	Comparison articles in trade journals such as PC Magazine would fall here. There may be some bias toward reviewing only products that advertise in their magazine, but comparisons are usually quantitative to minimize reviewer bias.
4	Non-commercial sources that still have an ax to grind (Consumer Reports is less biased than Motor Trend because of lack of advertising, but may be biased towards gas mileage and against horsepower as to important comparisons)
5	Includes non-commercial web sites and journals that accept no advertising. The article must discuss competitive products and be specific about advantages and disadvantages of products. Most handbooks, textbooks, and refereed journals are here.

Thorough

Thoroughness is hard to rate without significant experience. Therefore, for your purposes, this will have to be a relative rating scheme, and you are going to have to look at a lot of sources before one can get a high rating. One heuristic that you can use is “Would you recommend that the readers of your work seek out this source, or would you link it to your own web site on the topic.

Score	Thorough
1	Sketchy information/ no other comparisons/ would not link to web site
2	Best of at least 3 similar sources, probably would not link to web site
3	Best of at least 5 similar sources, may link to own web site
4	Best of at least 7 similar sources, would probably link to own web site
5	Best of at least 10 similar sources, listed on other people’s “best of” lists, would definitely link to my own web site.

In-Class Exercise

Exercise 1

Form into groups of 2-4

Exercise 2

Form into groups of 2-4

Exercise 3

For the surviving hypotheses from Exercise 2

Feedback Form

Name _______________________

1. At the outset of this unit, place a “B” in each category to indicate your self assessment of your initial, or baseline skill level.

2. At the end of the unit place an “A” in each category to indicate your self assessment of your skill level after practicing the skill. Be prepared to provide documentation for your assessment.

Novice (less successful)

Beginner (shows few expert behaviors) (1-2)

Good Start (some expert behavior) (3-4)

Getting There (many expert behaviors) (5-6)

Almost There (mostly expert behavior) (7-8)

Expert (shows all expert behavior) (9-10)

Expert (more successful)

Reflection of the Listener

What did I learn from this?

Which of the skills do I do pretty well? (List Evidence)

Which skills could use some work? (List Evidence)