Apple’s 100 -Tester Rule for iOS Apps Not Enough
Developers of all shapes and sizes have complaints about the rules and procedures put in place by Apple. One of the most frustrating is the 100-user cap for testing of new apps. This means that any new app can be tested at a beta level on a maximum of 100 devices that are pre-registered with Apple.
According to the Wall Street Journal these limits are prohibiting good development, especially for outfits such as Instagram that feel they require more testers and are gaining them by circumventing the system and just registering for multiple developers accounts (with each getting 100 testers). An expensive band-aid solution, but (at least for the moment) it works.
Apple claims they more or less arbitrarily chose the ’100 user’ number because it represented a large enough sample to ensure meaningful feedback but isn’t so many testers that it becomes difficult to manage. I think we should be a little more realistic –Apple set a limit because they don’t want developers just creating apps, rolling them out to ‘beta testers’ and never satisfying the App Store conditions such that they can keep the app in perpetual development but still in use.
There is a solution that works for some as Apple has implemented an enterprise developer account that allows organizations to roll out apps in a test-manner to their employees without those individuals counting toward the 100-user maximum. The cost is higher for these accounts and not just anybody can get one though, something developers think is unreasonable.
As with anything, restrictions inspire innovation which means other third-party solutions such as those that allow you to publish iOS apps as web apps for unrestricted testing through the Internet. While I can’t speak personally to how well those services work I can say that workarounds are rarely a good idea and usually come at extortion-level prices… but it might be that Apple hasn’t given developers any choice.
So if 100 test users isn’t enough, how high does the limit need to be? Unfortunately when it comes to identifying app pitfalls and bugs it can be the first user to try it that finds them or the 1-millionth, in the software-world results are just that random.
