How many of the following Macworld benchmarks are synthetic?

MacWorld published a number of benchmarks. I'd like to know how many of them are synthetic. I have some ideas about this but I'd like to know other people's opinions.

Speedmark 5 Scores are relative to those of a 1.5GHz Core Solo Mac mini, which is assigned a score of 100.
Adobe Photoshop CS3 The Photoshop Suite test is a set of 14 scripted tasks using a 50MB file. Photoshop’s memory was set to 70 percent and History was set to Minimum.
Cinema 4D XL 10.5 We recorded how long it took to render a scene in Cinema 4D XL.
Compressor We used Compressor to encode a 6-minute:26-second DV file using the DVD: Fastest Encode 120 minutes - 4:3 setting.
iMovie HD In iMovie, we applied the Aged Film Effect from the Video FX menu to a one minute movie.
iTunes 7.7 We converted 45 minutes of AAC audio files to MP3 using iTunes’ High Quality setting.
Quake 4 We used Quake’s average-frames-per-second score; we tested at a resolution of 1,024 by 768 pixels at the Maximum setting with both audio and graphics enabled.
Finder We duplicated a 1GB folder, created a Zip archive in the Finder from the two 1GB files and then Unzipped it

http://www.macworld.com/article/136214/2008/10/macbookbenchmarks.html

 

synthetic benchmarks are something like speedmark. the rest arnt synthetic.

I may be wrong, but I think synthetic benchmarks are benchmarks that dont actually do things. photoshop, and the rest, youre actually doing something so its not considered synthetic.

again, i may be wrong, but i think thats how it goes.

Testing quake 4 is kinda an old game, but as far as mac games are considered i guess there arnt that many, and they might be using it to get more favorable results compared to if they ran a newer game.

But yea, i think the results are to be expected, because they increased the cpu speed of the new laptops, so they should do tasks faster. Although im kinda suprised that the ddr3 didnt affect the results much, although again theyre mainly doing cpu intensive tasks.

 

Personally its irrelevant. Benchmarks synthetic or otherwise are just there to provide a standard on which to allow the comparison between to variables.. ie different laptop models.
Their point is to show or allow differences to be seen.
Thus they all serve a point and are equally valid (or invalid).. depending on your personal bias of what is important.
eg gamers will look to 3dmark and fps as the most important benchmark.
a

 

The only true Synthetic benchmark is Speedmark 5, because all the others are real world programs.

 

I think you are right.

I've looked up the definition just to be sure.

"Benchmarks are designed to mimic a particular type of workload on a component or system. Synthetic benchmarks do this by specially created programs that impose the workload on the component. Application benchmarks run real-world programs on the system. Whilst application benchmarks usually give a much better measure of real-world performance on a given system, synthetic benchmarks are useful for testing individual components, like a hard disk or networking device."
http://en.wikipedia.org/wiki/Benchmark_(computing)

 

yep. only the first one.

if you are trying to figure out for yourself whether a benchmark is synthetic or not, here are some tips (or useful things to ask yourself):

are the units of measurement of your score quantifiable or not? in other words, are you measuring time or average framerate, or something like points...

points = synthetic
real, quantifiable unit of measurement = not

is the program used for benchmarking intended primarily for benchmarking?

yes = synthetic
no = not synthetic

non-synthetic benchmarks are easily the best. however, synthetic benchmarks are often more repeatable, convenient, and practical. often, synthetic benchmarks are the only two way for two different people at different times to compare computer performance.

even with 3dmark there has been a lot of confusion with regard to scores vs. resolution.

just think about how difficult it would be to benchmark cinema 4d across computers. they "render a scene". even if you could share the scene file, there are about a billion different configurations and quality settings that would have to match up. its possible, but not convenient or practical. and thats just for one benchmark. similar problems with photoshop and the other programs. and quake is its own story.

but obviously, if done properly in house, doing a variety of real program benchmarks gives the fullest comparison picture between machine performance.