No systematic (regression) testing done yet
Issue #126
open
Originally reported on Google Code with ID 126
We don't really do any really good systematic testing yet, especially regression testing
is hard because not yet automated.
We should introduce automated test cases. The only solution for web-applications I
know is Selenium http://seleniumhq.org/.
Once we have a good base of test cases, it is a lot easier to check whether a certain
change breaks something (else).
And I think Selenium is really cool I'd really like to use it for this project ;-)
Reported by crazy4chrissi
on 2012-10-30 23:19:32
Comments (7)
-
-
reporter Although normal "non-web-testing" is fine to test a bug part of the code, some things in the end cannot be tested. The interplay of JavaScript and PHP for example is almost impossible to test. And all the other UI-stuff is a lot easier to test with browser-based tests like Selenium. Of course we could have unit-tests to test our classes and Selenium-tests to test the UI. But we need to restructure the code quite a lot before we can do good unit-testing I'm afraid.
Reported by
crazy4chrissi
on 2012-11-22 22:47:48 -
reporter Lol. "a bug part" was meant to be "a big part" ;-)
Reported by
crazy4chrissi
on 2012-11-23 21:11:39 -
Just noticed that PHPUnit can work with Selenium¹. I was then going to suggest PHPUnit as a testing framework, but the latest versions requires either php 5.3.3 (3.7) or 5.4.7 (3.8). It's actually becoming difficult to find non-namespaced code (< 5.3). Until now, I've tried to keep everything 5.2 friendly, but it would be easier to have different version requirements for main (single file/split file) code and code that is only used for development or support (build script, tests). Preferences? [1] http://www.phpunit.de/manual/3.7/en/selenium.html
Reported by
dreadnaut
on 2013-04-06 14:12:01 -
reporter I think it is fine to require php 5.3.3 or even 5.4.7 for test scripts. The problem with php versions is that you cannot update php yourself _on shared hosting webservers_. But you usually do not _develop_ code on a shared hosting webserver but on a localhost. Therefore, you can easily update php yourself in your development environment. So in my opinion, it is definitely okay to require recent php versions for test scripts. I mean, you can still modify phpliteadmin yourself with older php version, you just cannot run the tests. I am not sure if the build script is exactly the same. Although only used by developers, not being able to run the build script means that you'll have a hard time modifying parts of PLA like the JS. I'd have a better feeling if the build script did not require php 5.4, but at most 5.3. Otherwise we might stop some people from modifying PLA. PHPUnit definitely is the right way to test php units. I did not know so far that it also supports selenium. This makes it even cooler, so we cannot only test logic units with it but also test GUI including browser behaviour.
Reported by
crazy4chrissi
on 2013-04-06 14:25:30 -
I'll start using PHPUnit for new classes then, we'll see how it goes. In case anyone in interested in the setup, I'm installing it through Composer and keeping tests files under 'trunk/tests/'. composer.json { "require-dev": { "phpunit/phpunit": "3.7.*" } } In the future, I won't commit anything under vendor/ to repository. [For fun] We could also experiment with BDD and Behat (http://docs.behat.org/) which also supports Selenium :)
Reported by
dreadnaut
on 2013-04-06 15:02:41 -
reporter Okay. Let's see. I think when I find the time, I will need to create a test suite for the alterTable function (currently part of the Database class - famous issue #12). When I modify it, I always fear that changing a regex slightly might crush other cases which I currently don't have in mind. Having a good test suite would be really helpful for regression testing there. In fact I consider this method the only one in PLA that really has some kind of complicated logic ;-) Test cases for this should be easy to create, a CREATE TABLE statement of the initial table, an ALTER TABLE statement for the alter operation and a couple of post-checks, mainly comparing the actual result of SELECT table_info() with the expected one.
Reported by
crazy4chrissi
on 2013-04-06 16:00:35 - Log in to comment
Reported by
dreadnaut
on 2012-11-22 19:54:38