NYCPHP Meetup

[David Krings]

On 9/14/2013 2:58 PM, David Roth wrote:
> What packages are people having good experiences with to do web browser
> automated regression testing? I'm interested in this for being able to test a
> website mostly to make sure all it's functions are working for the user
> experience. Thanks!
>
> David Roth

Hi!

That is a loaded question for the following reason: in order to secure optimal 
user experience you need to run manual or semi-automatic tests. I cannot think 
of a single automated testing tool that can check for properly alignment of 
controls, etc. Humans are much more efficient and faster for such tests, even 
for regression.

Test automation is useful to test the backend for features that are unlikely 
to change. Do not use automation for anything that is still under development 
because you will spend more time fixing your test scripts than what you gain 
in test automation. A while back I gave Selenium a try and found it to be OK, 
but since I work in a Microsoft shop we use Microsoft Test Manager (which 
works fine with IE and sucks for any other browser). Microsoft Test Manager 
also needs a Team Foundation Server and if you do not have that set up or are 
not willing to spend a lot of cash it is out of the competition here.

For a while I also used iMacros for Firefox to automate a few processes, but 
mainly because I was lazy 
(https://addons.mozilla.org/en-US/firefox/addon/imacros-for-firefox/). The 
biggest problem with iMacros is that it tremendously slows down the response 
of the system. As long as you do not take iMacro's script run times as 
performance indicator you can quite easily click together scripts and run them 
as needed.

One other popular tool is Fitnesse (http://fitnesse.org/). I have read plenty 
about it, but have yet to find the time to dive into it...maybe I get to it 
ten years from now.

How large is your application? If it is under about two dozen features I 
wonder if automation will really benefit you. Keep in mind test automation 
means writing more code to test code, and yes, test scripts can have bugs as 
well! Also, test scripts will work fine as long as the test engine can find 
the control on the page. That means the automated test will not fail when the 
controls are splattered across the page or all on top of each other. Just 
because an automated test passes does not mean everything is peachy, all it 
tells you is that whatever worked before appears to be working fine right now 
(which is important). You still need to do some manual testing which will also 
reveal a bunch of issues the script wasn't programmed to look at.

Hope I could give some pointers.


David