Best Practises - Thoughts, Purposes and Benefits/Disadvantages - Testing (part 2 of 3)
June 21st, 2009on a train back to london, from the ed film festival, and i feel rejuviated i’ve regained some of my mojo after the hard slog of vr, time to talk about testing (for the record probably my personal weakiest best practise).
Testing
I’ll be talking a bit about testing, i’ll go through what i think the purpose and spirit of testing is, what’s it’s advantages/disadvantages, and then finally give a general overview of it.
Through out the 3 part brain dump on best practises i’m intending to try and stay constructive and positive, this is the one i may drop some hints of the inner scars i have on this particular one, sorry guys here we go…
Purpose of Testing
The purpose of testing to check the quality and reliablity of something, to do this programmers have developed a series of techniques and practises to help faciilitate this goal,
Types of Testing (by no means a complete list)
regression testing - this isn’t a specific technique, but it’s a type of testing, this is specific testing known to previous working areas, and checking to make sure nothing previously working has been broken,
Todo a regression test you would use techniques below after you’ve made some changes, and test the system to make sure nothings been broken by the code changes.
It’s extremely useful to have regression testing as it’s soo easy to break things while just tweaking a few variables.
integration testing - this isn’t a specific technique, but it’s a type of testing, this is testing when bits of code are added to code base (typically after a feature branch merge), this checks that all the bits of code are talking together and working happily,
Todo integration testing you would use techniques below after you’ve added a new feature (making sure the new function was working properly)
verification testing - different from scientific verification testing (which is a very worth while idea of having a test for your tests to make sure your actually testing isn’t wrong, and prevnting overfitting in my opinion the biggest issue i have with automated testing but i’ll get onto that), verification testing is basically a used as a quality control measure, using the technques below, you test your application before it goes onto the next stage of production, eg the stages could be say
build/testing -> integration -> release
This prevents rubbish code from say being merged into working branches, or even worse being passed onto customers, and it generally prevents time being wasted with bad code.
Testing Techniques (by no means a complete list)
unit testing - this is a type of testing which checks an small part of code, maybe a function or small task, this unit testing code is typically in a seperate file to the actual codebase, and these unit testing functions (typically 50% or more of teh size of the codebase) are all run together by a framework which produces a report ondemand.
The wikipedia link entry gives a wonderful low down on unit testing, remember unit testing code can have bugs too!
story driven testing - this is a testing technique i’m very new too (haven’t managed to use it in anger on a project yet), basically you have a application “drive” your application through use case scenarios, eg logging into a webmail app and sending an email to joe.blogs@hotmail.com, this means you can be fairly sure point to point, that your application works from a users point of view, (eg bugs like buttons not working get caught).
todo……-> rough draft for those interested
black box testing
load testing -
hand cranked testing -
Things to watch out for
race conditions -
bugs cluster -
multiple platforms/internet connections -
Advantages
on large projects can save time
reduces maintance bugs/mistakes
protects code from newbie errors
regressive testing can save a hell of a lotta stress
verfication testing can save blushes/time
integration testing can save alot of time, and reduce team friction
Disadvanatges
Over confidences and his brother overfitting -
Proper Unit Testing takes inital time reduces architural flexiblity
Bad testing can bloat code improves additional points of error
False positives can waste resources
Verfiication coding if too strict can be a bottle neck for some project (should i mention the political dangers maybe…)
Overall
Good testing massively improves software reliablity, stress, and maintance
Bad testing worse than no testing
All the automated testing in the world are poor subititues to actually checking a product by hand, (uniting se’s quote)
Best Practises - Thoughts, Purposes and Benefits/Disadvantages - Documentation (part 1 of 3)
June 13th, 2009Hi guys,
I’ve been thinking up some ideas of recent about how we (eg myself and the department) do code, it’s part of a planned initiative to do best practises and stuff, rarararra
In my opinion there’s simple stuff like use version control, don’t use eval and such, but there are sort of lower down, plain coding competency,
It’s not really best practise just really stupid not to do it, best practise to me is three tricky issues, documentation, code testing, and peer coding practises (by which i mean peer programming, peer code reviewing, peer branch/merge control techniques etc)
These are the hard and hence (in my opinion) interesting best practise areas, many common solutions cause as many problems as benefits, and all are open to bad implementation; as a youthful coder moons ago i remember a story told to me of an outpost of ibm’s bug fixing testing strategy, they awarded bonuses based on bugs detected[through unit testing] and fixed, of course this meant many of the worst coders who created the most bugs, got the largest bonuses, hardly a fair or constructive policy though perfectly well meaning and reasonable in theory.
Documentation
So onto Documentation, my personal favourite of the best practises (by my own definition of them), documentation comes in at several levels in a project and has a simple purpose
Purpose of Documentation
The purpose of documentation is to communicate something to someone,
Types of documentation (by no means a complete list)
- inline code documentation is an attempt by the author of a piece of code to explain the workings of that piece of code to other programmers and possibility himself,
- api code documentation (normally automated from inline doc code) is an attempt to provide simply explanations how to use code, and the risks, flaws and issues surrounding bits of it.
- code examples are normally for library code, these implement features using the authors code, and are often the most useful thing for other coders which need to use an library.
- wiki system documentation helps explain the overall picture of what a piece of software does and how it’s various modules and components work together, it can also on demand show dependence’s, peoples responsibilities, and other useful team info.
- bug tracing systems (such as Trac) help a team get a hold of the bugs in the system, tasks still to be completed, and allow test teams to inform the code team passively of flaws in the system without too much interruption, and alert them of the priorities/symptoms of these said bugs.
- changelogs in software allow for users/customers to see what has changed, and serve as a useful hint for testers to see areas which could potentially contain bugs.
- and lets not forget that a type of documentation which is often over looked is the simple spoken word, having knowledge troughs, information exchanges, or whatever word your particular management likes or is in vogue. Remember the bandwidth of real-life is massively more vast than that of text or audio, sometimes the simplest way to explain something is with a pen and paper (remember those things, i call them word 3.0).
Disadvantages of Documentation
- Documentation takes time, yes believe it or not writing words takes time, and everytime you update a bit of software it takes additional time to update it’s documentation (presuming your actually remembering too!)
- Bad or out-of-date documentation, this can be hard to read, bad spelling/English, and result in a “best practise” which doesn’t only not help anyone but can mislead them and actually be a liability to a project.
Benefits of Documentation
- Improved team communication from integration/system wikis/trac
- Reduced maintenance times, with documentation to help, implementation new features and picking up unfamiliar code should be quicker.
- New recruits will be able to get upto speed quicker.
Overall Documentation
For me documentation is a must have for any project, though others will disagree, Trac isn’t always needed for small projects but for medium to large my god does it become useful (as long as testers stop double-triple submitting bugs!!!), inline code docs/examples and apis are similar useful, with apis and examples being a MUST MUST have for any library, and inline code being a useful habit more than a must have. Changelog are a personal preference i like the ability to be able to justify myself using a changelog, and lastly what all of documentation is about,
Documentation is about communication, don’t underestimate simply talking within a team to solve problems better than email, apis, wikis, and Tracs. We all have a beautiful language and mother tongue, we should use them more.
Testing and peer practises to follow would write more but it’s such a lovely day…
Hacking Highscore boards, it not big or clever, but it is fun
June 12th, 2009Normally I’m quite a nice bloke and play games without cheats, but some times a game breaks me and i need to prove i can beat it,
so i go into tamperdata and give myself some silly score - 69/666, etc
here’s one i did now, (I blame low blood sugar levels or something)
In a crazed moment of madness, I realised all games were at my mercy i could terrorize them all, ZaPPEr69 RuLZ etc etc,
I pulled up to my favourite long standing bear bug game, gemcraft chapter0, think it’s clever to write open ended games with a high addiction factor and no way of winning for me to get my fix, i’ll get you!!!
i setup a fake facebook, i analysed the post data, it was all falling into place, dead easy…
Mwhahahahhaha it’s mine it really is, i’ll get you all, for this insult to my man-hood, don’t think i have a great score, mwhahahahhaha …… I’ll get you and your little winging monkey,
At this point a saw the saddest thing inbetween the post data, the saddness wee Server Type I’ve ever saw and maybe ever will, “MochiWeb 1.0 (I’m not even supposed to be here today)”,
that poorly underpaid, under appreciated little sysadmin saved mochi from the worst hackage they would of ever seen, I just couldn’t bear inflicting pain on a human being, servers yes, but the human cost of hacking them all would be annoying, irritating to this poor sysadmin, obviously not having a great day anyway (and after the week I’ve had I can relate)
Incidentally , having zero autheication on a completely open http post, with non-hashed data, and the game id that is a incremental number is a bad idea, i was about to submit 666,666 for the first few hundred score boards.
Bless I thought, the anger passed, maybe I’ll just put down the keyboard and continue my life, you know the one outside, the End
Zap
Three wee things about bbc news
June 8th, 20091. oops, bbc news team inserting placeholder text into cms
2. Why is opera-mini inserting google adwords into the bbc news website!!! grrrr
update -> cool online opera-mini demo may setup a version of this software for the visualising radio mobile version
I love opera-mini but i wish they wouldnt do this shit (prosuming it isn’t O2), having said that the adverts look perfect and look almost designed to fit into the place they have inserted them into.
Sorry for the poor image quality since i use my phone for all my camera needs had to use the iMac Camera which is rubbish.
3. Finally the lord of darkiness, lord mandelson, wow what a man, no one can deflect, lie, or weasle quite as good as mandelson,
points to watch out for are :-
the amazing deflection and complete non-answer of the first question of the interview (first 2 minutes),
the proceeding demonition of the poor bbc anchor who has the interview dicated to him by the frankly amazing misdirection, deflection, and various other straw man techinques from the master of the dark arts,
then my favourate highlight is in the point where the whole operation is undone by the slightiness freudian slip of the tongue where we hear something quite unbelievable, simply amazing at 17.55-18.25
sorry for going into politics, i don’t normally infect my blog with this stuff but this guy’s brillant, it’s the most interesting thing i feel watching politicians do what they do best: bending; maniupating; stretching the truth.
Like I said this a point and case study in the dark arts of the politician.
And now for something completely different
- my favourateiest bestist new song ever! since i first heard it yesterday
- the best source of all knowledge on this planet, no seriously i mean it
thanks all folks
Zap
whoope! - radio vis mark 2 is up and live!
June 1st, 2009after 7 extremely hard weeks, and one of the more stressful dress rehearsals of a my career with the 5live team , watching mark kermode, and the shorter bloke from the peep show appear on the radio vis client was amazing,
Visualising radio mark 2 has launched, it’s a bbc experiment to see what can be done to engage with audiences more, personally i like to think of it as web red button, it’s only a trial but the tech is there to do some really cool experiences,
wanta show the wikipedia entry of outcast as their playing, sure….
wanta show a rss feed of premier league goals, sure…
wanta build a module to dynamicly parse the football press acc ’s feed and push it into the client, sure… will cost a bit though
wanta show the music video, sure… though the trust would probably never allow it
it’s a modular system easily expandable to test out how to do this thing, and it’s great, but only the start, only a fraction of the potiental has been reached, the ux ain’t perfect yet, and is even a little confused in places, are the audience passive or active, should we be pushing or should they be choosing, the important thing is the base tech is here now, with it’s extendable api mark 3 could really be something special, and i mean that, we just need the courage to build ontop of it,
today’s launch was no less amazing, alot of flash/ruby/monterosa tech went into this thing,
sage stats, rss feeds, webcams, sms messages, vcs feeds, drm(evil…. but necessary oh i feel so dirty), json, nuts, bolts, and even a few big name celebs
it’s going to be a fun few weeks
started today it, it’s to be repeated for the 5live simon mayo show 1pm to 4pm, monday to friday for the next few weeks, then expanding further into other networks
http://bbcami.s3.amazonaws.com/live_5live.html
for those of a geeky persuasion here’s the debug link (no secret easter eggs i’m afraid but it shows you a bit more about hows it’s been made)
http://bbcami.s3.amazonaws.com/live_debug_5live.html
ondemand links (of questionable if strongly debated quality) will be on the website soon
i can honestly say, with the restrictions, time-restraints, set-backs, this is one of the proudest products i’ve made, hope anyone watching it enjoys it, will probably do a bbc radio labs blog posting about the vis radio client/console at some point its a ugly disgusting, thing of beauty , most of the credit of which has to go to the the mark 1 team,
cheers
Zap
updated to include the<censored>radio blog post</censored>
Weddings and Geeks
May 2nd, 2009
Helen and Rob at their Wedding, plus the flower i managed to catch, which kinda annoyed the girls....oops
Just been at my friends helen and rob’s weddings it was lovely, a truely beatiful event which was magically and special, these are the first of my friends my age to get married, and it seems everyone else is doing it too, with another 2 planned for the summer, and everyone else having babies aside!
So without going into my singlehood, and all my friends being couples etc etc, and the fact i caught the flowers at the end of the wedding, and the amazing goth wedding band… blah blah blah
I spoke to a guy who explained the 3 actionscript languages in the best way ever…. no really
Zap -the Warrior
April 8th, 2009my settings from this game
What Beauty is there in a City
February 7th, 2009The Dangers of Search Engines and Flash … and dummy data
December 23rd, 2008I was surfing around one day after my office work party, in a perfectly normal bit of narcissism I decided to Google my name, other than Google rather annoying trying to change my Scottish name back to the Irish spelling, I found a rather interesting new entry…
Yes it seems Google has indexed the main swf from the project I last worked on during “electric proms”, basically I had some dynamic text fields with my email and name as the default email sender, obviously this is data overwritten at run time when not in debug mode,
Indexing swfs is not a bad thing other than the swfs are useless by themselves:
- swf general do all their relatively links to files from the html location rather than the swf (mine do the later incidentally…);
- without query strings swf’s often break; swf are normally not a page in themselves (at least they definitely shouldn’t be!) without the context of the html alot of meaning is lost;
- the random way the dynamic text fields are being indexed means they have virtually no semantic meaning by themselves in any case; perhaps of course this is just another reason why careful tab indexing is so important in flash
- and so back and forth…
In short if Google had condensed the content of the swfs and then added this metadata to the combined pool of metadata for a given html page that of course would be great, but of course indexing swf by themselves is silly, plain plain silly… pointlessly silly
But apart from indexes swfs without the context of their surrounding html being a bit daff, they are some guidelines I’ve thought about to make sure I don’t do this again in future, in this rather nasty brave new world I’ve found myself in.
- If you have sensitive information inside swfs as dynamic text fields remove them now!!!, this information can be indexed the same as can info on an unprotected html page
- All those debug text fields off-screen and such, make sure they are populated with a blank strings, obviously this is a lesson I also have to learn…
- (This one is from here) Google says they index any links found inside a swf, obviously watch out for code executing pages, prize winning pages or naughty debug hidden pages as these could be indexed and executed from the Google search bot, eg http://www.blah.com/delete_all_info.php, http://www.blah.com/hidden_page_youve_won_an_iphone.php, don’t add any links into a swf which do this
And lastly having read a few blogs on the issue I’ve found some stuff to watch out for:
- This is one less reason people have not to build all flash website, if a producer asks to do the much unadvised all-flash website approach, the much relied on Google does index flash argument doesn’t work any more. However, No point labouring the point like some, best to approach the point slightly more so
- Google have stated that they are trying to index any xml/data files accessed by any indexed swfs, so be careful in what you put into any xml files your swfs use!
Essential this whole experience has added to my growing suspicion that flash is an immature platform in so many ways, this is just another flaw to add to the list along with accessibility issues, dependable platform backward compatibility…
In short I’d recommend everyone adding this line to their robots.txt as I don’t think flash indexing is finished cooking yet
Disallow: *.swf










