Thursday, 19 March 2015

Why API design matters



Disclaimer: This post is not all my own work.

I recently had to take a look at API exposure in user-created Powershell modules and in a new internal programming language that uses XML. I found that a lot of my conventional knowledge around what makes programming easy and bugfree was not being followed. Why is not the concern, but how was. So I have copied and pasted and referenced and linked and pasted more and re-organized and added some of my own stories to other peoples work below.


So why does API design matter? [R1] 

As someone who has to consume APIs from lots of places, the APIs I’ve been most productive with, follow a few rules. Here are some I’ve picked up over about 20 years.

Please write your API in a consistent manner:

  1. Consistent naming within the API itself. Use verbs, nouns, keywords in EXACTLY the same style or way all of the time. 
  2. Consistent with the target environment it will be used in. Take the programming language into consideration, python, Powershell and so on. If it’s on .NET, then consult Microsoft's naming guidelines.
  3. Consistent concepts. Factory pattern? Builder pattern? Static methods? Interfaces? Just pick one, and stick with it. REALLY. There is no such thing as a small exception to the rule. It will stick out as a big sore thumb. More than 1 exception? Your API is more and more amateur.
  4. Make it easy to figure out how your API will work. I use APIs from lots of places try not to assume I remembered that your personal preference for naming widgets changed at some point from always having an ‘s’ on the end, to dropping the ‘s’ later on when the last few functions got added. This goes for named arguments too naturally.
  5. Save yourself on documentation and the iterative code->debug->refactor and repeat steps. There are so many great pointers on the actual API development steps in this slideshow, that I'm going to just link to it instead [R3]

Here's another one: Specificity.

  1. Return types. Functions often return true false or an integer that indicates a code, or they return data. I have a pet-hate for the function that falls into the second category, but lets move to the last category.
    For example. When a function will return some data, don’t just say it returns Objects, and then leave me to guess what form those objects might take. Example: can I expect a, string, array or an object that has specific properties. I cannot know if your API is useful to me if you don’t tell me what properties it is going to return to me until I actually call it. Then going further, nothing guarantees that depending on the inputs the output may have varying sets of properties. Say for example I use your Get-widget(‘patrick’) to return a record about Patrick the starfish
    username=patrick
    userid=42
    shape=star

    But how useful would it be to know that for example if I said Get-widget(‘spongebob’) would I expect this for example to get returned?
    username=spongebob
    userid=1
    porosity=42

    Now "I" expected that to come back, because Spongebob is a sponge, but you didn’t expect that. So what a function returns is really really important.
  2. Naming
    Please don't get silly-long with names: StartProcesswithRedirection. You can often refactor super-specific things into a more generic name + parameters.
  3. String parameters for constants are often evil! In Powershell its often convenient, but when it is possible use enumerations.  I'd much rather like an enum I can investigate: or a Get-RedirectionEnums() ‘none’,‘stdout’,’stderr’,’both’. Now documenting this is all fine and well, but if in a future API you decide to deprecate ‘both’, and require people to use something else, all they have to do is call the RedirectionEnums() command, they don’t have to read the docs, and you don’t have to update your documentation in very single damn place where you copy/pasted the above values. Smart?

Be complete and verifiable

  1. Verify: There’s nothing worse than pulling a trigger and then thinking, hey! Did that work right or not? I believe that for meaningful software to be written using any language (or API) every operation to modify the system needs to have a similar function to restore the system state or at very least to probe the state. This means New must always have Delete or Dispose. Add must have Remove, and so on and so on. To not have this causes API orthogonality issues and blocks many use-cases. What I’m talking about is not new, See chapter 4, The Art of Linux Programming 2004 [R2] .
  2. Orthogonal: In general to myself it makes more sense to design for and create orthogonal APIs so that not only can all future workflows be done with your API, but also the API be fully testable and stable. That way I never have anyone coming back and saying things like:
“Hey John, I need a feature to be able to move a widget from one folder to another. Can your API do that?” My response must always be “Yes Spongebob!”, because it's only 3 lines of code:
1: create blank object in the new location
2: copy the old object into the new blank location
3: delete the old object
I don’t have to change my API, and the user does not have to write more than 3 lines of code.
 3. Symmetry and New/Delete: It also makes sense to me to always implement an undo for every action to allow for cases where an operation fails for any reason. Depending on the API or service, and the point that it fails, it’s otherwise impossible to know if the operation is half-complete, or got rolled back or just never modified the system at all before failing. Pairing up getters with every setter and New() with Delete()/Dispose() makes this question go away. It makes automation and remote control under failure scenarios really easy. If an API does not support rollback/undo operations, you want to often be clear why an undo function is not necessary. For example there is no function to uninstall Windows 7, why? Well because not only is it not functionally useful, and format.exe would have the same effect, but the workflow and user experience is that you only ever want to uninstall in extreme corner cases. Microsoft want you to pay the license money and stay. If your API does not need undo(), either explain why it does not, or we make it clear in the API design. Symmetry is everywhere.
4.
Works in progress:
In cases where implementing all api methods to complete a pattern, is not realistic due to time constraints un-implemented function stubs can be used to allow static analysis of the API surface. This process of stubbing in time-bound or agile development is typical TDD process. Not closing or making an API encapsulate it’s purpose fully, conversely leads to accidental functionality leakage into other pieces of software which either introduces duplication or interface sanity bugs. 
5. Easy to learn: 
Compactness or completeness of anything programming or real-world,  as with APIs is about the sweet spot that allows you to not have to worry about external systems that you know nothing about that lie beyond the API or interface that controls them. For example you don’t have to know how an AM/FM radio works, to get sound on either an AM or an FM frequency. Good APIs meant you only need to worry about inputs that have meaning to you the user of that API. If a specific implementation of the radio class happens to have an API that tells you the signal strength or some other technical details, well that’s totally optional and does not interfere with using a radio in general at any point.

My Creds

So you may ask why Conrad are you qualified to sprout all of this? Well, that's because I once wrote all of these drivers against an interface that never ever broke once over a period of 12 years:
Asea Brown Boveri Procontrol 214,Allen Bradley SLC500,Allen Bradley Ethernet TCP/IP,Allen Bradley DH+,Centralised Fire Panel 2000,Conet Exception-based,ConetPCI Exception-based Driver,,D Le Roux & Associates MUL-T-LINK 8.1,,Eagle PC Card DRiver,Eagle EDR-Enhanced (EDRE) PCI Card,Electromatic Optolink V24 Dupline 128,GSM SMS Driver,GSTProfi Exception-based,GST Profibus SoftFEP,Hitachi Ethernet,Hitachi H,JOYSCC Exception-based,Koyo DirectLogic-105 Range K-Seq.,Koyo DirectLogic-205 Range K-Seq.,
Klockner Moeller SUCOM-A,Koyo DirectLogic Ethernet Protocol Driver,Serial,Mitsubishi AJ71UC24,Mitsubishi AJ71E71 TCP (not UDP),Mitsubishi FX TCP (not UDP),Mitsubishi FXS,Mitsubishi Q/QnA Ethernet,MIT_M2M,Modbus I (RTU and ASCII),Modbus Plus (SA85),Modbus Radio,Modbus Ethernet,Moore ICI 320,Omron SYSMAC,,Opto 22 (OPTOMUX),Pager Exception-based (Discontinued!),PolyComp Sign
(GTX),PolyComp Sign Ethernet,RDC8102 Exception-based,Sascom (RTU and
 ASCII),RS-232 Barcode Scanners etc. (SCANRS),Schiele SYSTRON S 800 3964 and 3964R,Schiele Custom Protocol #1,Siemens S7 Funtions,SiStar PCU Server Driver (Intel),SpaBus,Spectrum Tele-RANGER,Spectrum SCADA-MUX,Square D Sy//Ma
x Serial,Strike ENERMAX,Strike ENERMAXP Serial and Ethernet driver,TDC/I/0550 Serial,Texas Instruments 500/505 Series,Texas Instuments TCP/
IP Ethernet,TOSHBIN (RTU and ASCII),Toshiba (RTU and ASCII),Toshiba T2 Ethernet,Yokogawa uXL DCS.
And those are just the ones I wrote, customers added quite a few, and the full list is double this size.
Basically I helped publish the API and we sold the spec as a public API along with some demo code. What's remarkable for me is that the API started as 14 functions, it expanded to over 21 before being completely re-written later. Moreover it was of the Service Provider Interface (SPI), where API breaks are close on 97% fatal, and so I suppose, that's where and why I learned what you can get from Google in 5 minutes now.

References:

Why does API design matter
R1
The Art of Linux Programming
R2
Why good api design matters : slides :Joshua Bloch
R3
Why good api design matters : video :Joshua Blochhttps://www.youtube.com/watch?v=heh4OeB9A-cR4
Image "How the analyst designed it" CC attributionhttp://projectcartoon.com/cartoon/3
R5

Wednesday, 18 March 2015

Sound Creative Commons CC

To download some CC sounds for foley especially, have a look here.

http://www.freesound.org/search/?q=&f=license%3A%22Creative+Commons+0%22&s=score+desc&advanced=0&g=1

I did say this blog was a braindump, until my next actual electronics related proejct with the raspberry pi, that is all you get :)

Monday, 16 March 2015

Live Wires excitement

I am volunteering to be a helper on a Scripture Union holiday camp called Livewires, it's a slightly geeky format holiday camp. I still have to pass all of the background checks, so I need to start that ASAP.

It will entail:
  • Prep weekend Friday evening to Sunday afternoon. 
  • The holiday itself is eight days in mid August. (1–9 August 2015)
Background check starts here : http://www.scriptureunion.org.uk/livewiresteam
Leaders will be doing : http://livewires.org.uk/helping
The cost for leaders in 2015 is £230


Wednesday, 18 February 2015

Programming introduction for kids



I have been increasingly aware that children are not getting a good introduction to the world of software development in school. Mainly because teachers are often not trained, or get limited by curriculum and resources. A few external ventures such as coding club (run by ARM volunteers) and a visit to the computer history museum might spark interest, but there is always going to be room for making learning fun. Programming might be an interesting hobby for kids, but is also a very useful way to get to grips with a technology area that drives a lot of what we do today. So I'll not bore anyone with how Bill Gates, or Mark Zuckerberg jump started their careers in software, but rather go hands-on with a practical lesson aimed at children 10-13 years approximately.
Aimed at those with no experience, to the intermediate, I am going to run a one-off coding club Saturday. If it works, we will repeat it. It will be a drop-off 3 hour session run by industry professionals and teaching kids the basics programming using Scratch on a Raspberry Pi.

Why 10-13? We have chosen our target age range, in the hope of getting girls as well as boys involved. I'm flexible on age, but find this is the age-range is where computers are cool and there are not yet any teenage distractions. Not too young to be able to grasp mathematics and to think logically either. Historically software engineering and many of the sciences have seen also few women taking up the career. I do really hope to get some of the boys who rage about glitches in Minecraft for example, to create simple programs of their own and perhaps go on to become the next Silicon Valley startup founders.
Why Raspberry Pi? Two reasons: One; is it's a cheap and small computer and can interface with the real world and bring programs to life. Two; it is very simple and does not have the distractions associated with programming on a desktop computer.
Just how basic? Scratch. Scratch is graphical, but has enough basic features of proper computing languages. So we can teach software engineering principles that kids can later take into careers.
Who? I'm looking for women. Yes women in software industry who have a Saturday spare. At the moment, Rob Pearce, David Newton and myself Conrad Braam will be running and helping put together the content. If it's popular a repeat Saturday would benefit from having a practice run, and have girls come along. But for that to work I need a female volunteer.

As a takeaway, kids will have learned how to use Scratch (some already have experience using it) to create a simple computer program. They will learn programming concepts such as design and debugging from people who do these things every day. They will learn how to link programs to simple hardware attached to the Raspberry Pi. And they will create a small computer game, who knows, sometimes that is enough to get them hooked. It was enough to get me hooked back in 1985.



Because Scratch is not a proper programming language, it is also our intention to hold a more advanced Python programing session. The takeaway will be the same, but will allow for more practical application of knowledge. Python is much more popular for use on the Raspberry Pi, and there are hundreds of good examples out there to get kids to go further. Which brings me back to the reason I'm promoting using a Raspberry Pi, they are cheap so won't break the bank. They are simple to use, and simple to dig deeper into when it comes to it. The internet is full of cool Raspberry Pi projects for those who do get hooked.

Charge will be about £7.00 for 3 hours which is a good deal really, and will cover us for equipment.
No need to bring anything with you either on the day, snacks will be provided. Kids take home a head full of inspiration. Location and date (before summer breakup) still to be decided.

Wednesday, 10 December 2014

Frog clip

I'm looking for a animation or actual frog on high speec footage clip. Green screen or contrast background preferable obviously.
All I can find is one clip that looks ideal.
https://www.youtube.com/watch?v=hnx8tHyKMNU but not shareable :(

Wednesday, 12 November 2014

Use Cyanogenmod and hosts file to block ads

I am running CM11 on a Samsung Galaxy S3. I noticed that add supported apps suck a lot of my data and are sometimes just producing plain buggy and annoying banners.
Block em!
ok So I googled it, a few decent hits are:
So let's try it out. But before we go, lets get our hosts list pre-compiled. I found more resources and a slightly old one here http://mathdotrandom.blogspot.co.uk/2010/12/block-ads-on-pc-android-with-uber-hosts.html

So far it's not blocking much, but it's a start.