Interesting Article: An attempt to test test kits.

This may be old news to some of you, but I've been catching up on a lot of hobby related reading lately and I came across this article on Advanced Aquarist:

Aquarium Chemistry: An Attempt to Test Test Kits — Advanced Aquarist | Aquarist Magazine and Blog

It's a very interesting read. It's not the typical, "which test kit is the best" type of article. The authors took a lab tested water sample to a reef club meeting. The club members were asked to bring whatever test kits they had at home to use for the experiment. They all took turns testing the water.

The variation is results is quite startling. Some people used the exact same kit and got different results. The number of people with expired kits is also pretty high. Check out how far some of the hobbyist's results varied from the lab analysis as well.

 

thats definatly interesting and really weird that that many people can screw up the tests that bad lol

 

well makes sense that they are mostly off. even if all samples were taken from the same container there is a chance you will get different results. just like a snowflake no two samples are going to be exact. other factors add in such as was the sample sitting still for a while with no movement, was it taken straight from a tank or did it sit in a container for 6 hours before being tested. So many different things could affect 2 seperate drops of water.

I remember reading once on qt tank setup that a single drop of water can contain billions of differents ions and other invisible pathogens. If you take that into consideration check your equipment, do you completely sterilize it after every use? Slightly dirty testing equipment can even throw your numbers off.

 

good point lol I just rinse mine out with sink water and roll ;D

however the tests were very similar when they did it between the two of them so human error seems to be a repeating factor.

 

what interested me the most, was the 1st comment below the testing

I cannot be certain, but I think Boomer William Wing, may in fact be Boomer (Bill Swann or William Swan) from RC , who if any of you have visited that forum some time ago, will recognise as someone who knows his stuff in terms of reef chemistry ( Boomer not as active these days sadly)

whoever it is, he does point out some errors in the testing methods, the standards being used, and the types of test kit being compared ( they are not all measuring the exact same parameter and mathmatic conversions need to be applied in order to get comparative figures)

Steve

 

yea thats another good point

this is alot of the reason I am going tot go to hanna checkers just soome like they will give a more accurate reading and to keep the human error down.

 

thats actually pretty scary how inconsistent some of them were. I understand that there may be user error but I think the whole idea was to test not only the kit accuracy but the variation of results with both the kits and the users. The most accurate kit in the world means nothing if the user screws it up. So when hobbyists post their parameters on the forum, it doesnt matter if it's the kit that is wrong, the user that is wrong, or perhaps a bit of both. What is important is that there is such a large variance in the results. I work at a LFS and I do water tests on peoples tank water all day every day. One thing that I have noticed with the API nitrate test kit, is that I can test the same sample of water multiple times and get different results depending on how long and hard I shake test bottle #2. Be honest, do most people REALLY shake that thing for 60 seconds? I highly doubt it. And with refractometers, some can be accurately calibrated with RO water and some can not. It's best to use a calibration fluid with a known specific gravity very close to the range of our tank water. One day at work, we opened up 5 brand new instant ocean hydrometers. 3 were dead on, one was .001 point to high (the control was a refractometer) and another was .002 points to low. Some may argue that .002 off doesnt really matter as long as it's consistent, but if I have hundreds to thousands of dollars worth of coral in my tank, I like to know exactly what my water parameters are. I have even seen refractometers be off by that much depending on the temperature and whether or not you hold the plastic flap down over the lens of just let it hang. I want to start messing around with conductivity meters to see if they are more consistent.

 

Boomer William Wing is not Bill Swan but rather William Wing (Boomer is his nickname). He is indeed the same Boomer on the RC Chemistry forum, same one as the Chemistry forum MOD on a few other forums. He's still extremely active, just not as much on RC since he has other forum obligations elsewhere now.

 

I was struck more by the "human error" factor in the testing. This was a reef hobby club, and many of them had expired test kits. Also, testing methods aside, there are a couple of instances where two different people got wildly divergent results using the extact same testing kit, and the exact same water sample. One example had one person getting a nitrate reading of 25 ppm with a Salifert kit and the other got a 100 ppm reading with the same kit.

How many hobbyist do you think may be wildly off in their assumptions about their water quality due to human error or expired test kits?

 

If everything in my tank looks happy, which is the case, I rarely test anything.
Lately only Ca and Alk., rarely.

Great article however!