Monday 2 March 2015

Correct By Design

Applying formal methods from the starting phase of Software Development Life Cycle ensures a product which is foolproof in all respects. This is what we mean as correct by design. Correct by design approach is different from formal verification which only proves that the product is operationally correct. Correct by design starts at the level of specifications. The domain could be hardware, software or genetic engineering (Lifeware). In the case of hardware, everything is clearly specified. The prohibitive cost of mistake in a hardware design has forced the engineer's to adopt rigorous formal techniques to validate the hardware. Formal methodologies have become mature and have become integrated engineering practices in hardware. The circuit produced needs to be correct, not because tests are run, problems are found and changes are made. It should work correctly the first time it is used (or tested) because it is designed and built right. Building right is neither an art nor a fluke, it is very much a mathematical activity. Systematic application of formal methods leads the designer to reach the same statement of the specification. The validation team then proves that the design meets the specification. In the critical cases, formal methods team and testing team take the same specification and meet the requirements of proving them correct. Genetic engineering is also following the same methods so that chances of error are eliminated.

On the contrary software development is asystematic and breaks down often in the absence of clear specifications. In correct by design approach, formal specification is mandatory for all applications. Having clear statement of the requirements leads to systematic and precise specifications. In the process of deriving precise requirements from loose requirements, incompleteness or contradictions get eliminated. When the precise specifications are achieved, it's complete and consistent in a mathematical sense, and thus traceably correct. It's easy to say whether a program is correct or not because the program has to perform according to the specification. In the instances of software development where specifications or statement of requirements are missing, it is very difficult to say whether the program is correct or not.

Software designers seldom put effort to be correct by design and to get it right the first time. This is because they want to keep the software flexible till the point of delivery so that any user requests can be accommodated till last minute. This flexibility is misused and often leads to poor quality of software. However, it is possible to achieve the same degree of correctness with software as with hardware. Software is no more complex than hardware. Software developers need to apply the same level of rigor of training and workmanship that is being done in hardware or genetic engineering (Lifeware). This would make software engineers develop products that's correct by design. "Good enough" software produced by hit and trial methods, testing and changing approach has to give way to correct by design approach.

Its a common notion that software developers can't produce software of high quality. Software glitches have become a common way of life. Be it mobile apps, banking solutions or web-portals. Glaring failures of ArianeV and death of patients due to over-exposure of radiation in certain medical machines have made everybody sit up and notice the software failures. The reason of often software failures is not spending enough effort and cost on the software design. Coding takes the center stage as a software engineering skill. To overcome these shortcomings in software it is important to spend as much money (effort and energy) up front for getting the design right. By applying the meticulous design process with exact (mathematical) primitives one can keep the impurities and errors out. The focus needs to be shifted from testing and release to up-stream for developing precise specifications. Code should be developed to meet these exact specifications. The essence is to follow processes that keep the errors out. The code thus developed is functionally verified to remove errors that were made during coding process only. Finally, testing is a statistical activity designed to demonstrate that there are no errors (rather than to find errors).

Another aspect is that engineers should be licensed and specifically certified for a particular domain. Compiler development, enterprise software development, mobile app development, CAD software development, medical software, embedded or mathematical software development, each is a different type of software from each other. One can shift domains if adequate skills and training is acquired. For example, developing a test tool is not same as developing a web-portal or developing a CAD tool is not same as developing a simulation tool. Formal methods are being extensively applied for control systems engineering and cyber-physical systems design. Many researchers are working on automatic synthesis of controllers from discrete specifications (languages, finite state machines, temporal logics, etc) for dynamic systems. The synthesized controllers describe both the control laws, discrete specifications as well as its “correct by construction” software implementation.

Correct by design approach is equally useful for simple as well as complex systems. Rigorous specification languages (Linear Temporal Logic, Computation Tree Logic), detailed design enunciation constructs(Finite State Machines, Binary Decision Diagrams, Multi-valued Decision Diagrams) and finally testing to prove the correctness, is the way to go. It will work the first time and every time. 

For views of Jesse Poore on this topic refer to following link:
http://ubiquity.acm.org/article.cfm?id=985613 

COTS Or Custom IT


Enterprise Suites or Customised Solutions

Attempts are being made in government sectors and various businesses to utilize the information technology to foster the profits and ensure the accountability. Case studies show indicative successes in businesses. Government sectors are yet to post glorious success stories. Implementation of IT based systems require deep analysis of the background of organisation vis-à-vis IT solutions being offered by the COTS solutions. There are innumerable vendors offering generic IT solutions to highly customised domain specific solutions. A family of solutions called ERP (Enterprise Resource Planning) solutions is the most prevalent buzz word in the industry. Many big to small vendors are offering ERP solutions with elaborate to limited features editions. The costing of each feature complex, licensing complicated and direct applicability unclear. It is time that we shall look at the options and solutions for enabling the organisations with IT. I ponder over various aspects of decision making for implementing an IT solution. The question of security of the system is not addressed in this discussion (requires a larger and individual space), though it is very important and is actually addressed when we initiate the implementation. It is mandatory to study the built-in features of a solution vis-a-vis domain and specifications before zeroing upon a solution. Let’s look at few aspects of decision making factors in thinking about IT solutions. 

COTS Solutions for ERP

The most popular buzz word in the area of enterprise IT is Enterprise Resource Planning (ERP). ERP is a business management software suite that an organisation or a business company uses to collect, store, manage and interpret data from various business activities, including product planning, trading, finance, manufacturing, service delivery, marketing, sales, inventory management, shipping, tracking and payment. The aim and goal of an ERP is profit enhancement and growth of any organisation. The resource being managed could be raw material, funds, material, inventory, products, humans, services, infrastructure and anything which is the core business of an enterprise.

ERP aims to provide an integrated view of core business processes, often in real-time, using common databases maintained by a database management system. ERP systems track business resources—cash, raw materials, machinery, man power and production capacity—and the status of business commitments: purchase orders and payroll. The applications that make up the system share data across the various departments like manufacturing, purchasing, sales, accounting, etc. ERP facilitates information flow between all business functions, and manages connections to outside stakeholders.

ERP is a concept. Hundreds of ERP types and solutions exist which are applicable to various domains, sizes and contexts of application. Primarily aimed at helping businesses to operate business efficiently, the ERP solutions also help to cut down the unnecessary expenditure, delays and wastage of resources. Once all these are streamlined, the business intelligence and streamlined control provides clear indicators of potential growth. Thus, practically all aspects of business are covered under ERP. ERP is not a fancy word for boasting that we have implemented ERP. An ERP implementation must show that there is a marked difference in the profits and there is a clear indication in numbers that there is return on investment. 

Functional Areas of ERP

An ERP system covers the following common functional areas. In many ERP systems, these are grouped together as ERP modules:
  • Financial accounting: General ledger, fixed asset, payables including vouchering, matching and payment, receivables cash application and collections, cash management, financial consolidation
  • Management accounting: Budgeting, costing, cost management, activity based costing
  • Human resources: Recruiting, training, duty rosters, payroll, benefits, diversity management, & retirement.
  • Manufacturing: Engineering, bill of materials, work orders, scheduling, capacity, workflow management, quality control, manufacturing process, manufacturing projects, manufacturing flow, product life cycle management
  • Order Processing: Order to cash, order entry, credit checking, pricing, available to promise, inventory, shipping, sales analysis and reporting, sales commissioning.
  • Supply chain management: Supply chain planning, supplier scheduling, product configurator, order to cash, purchasing, inventory, claim processing and warehousing (receiving, put away, picking and packing).
  • Project management: Project planning, resource planning, project costing, work breakdown structure, billing, time and expense, performance units, activity management
  • Customer relationship management: Sales and marketing, commissions, service, customer contact, call centre support - CRM systems are not always considered as a part of ERP system but rather Business Support systems (BSS).
  • Data services : Various "self–service" interfaces for customers, suppliers and/or employees 

Implementation of Enterprise Solutions 

Many ERP vendors offer plethora of suites to implement in businesses. Some are domain specific and some are generic offerings. Before we look at the ERP solutions applicability, we need to understand the three aspects of any ERP project:

Implementation: A process that defines the method to implement the ERP Software in an organisation. The ERP solution comes pre-configured means an organisation just needs to set up the master data (e.g. employees, materials, customers, suppliers, pricing records etc) and start transacting on the new system. In reality this rarely happens; as all organisations are different in some way or the other. Normally the standard settings are modified as per need (to extent of flexibility offered by the system).

Configuration: Most of the COTS solutions offer existing parameters and functionalities which can be assigned the organisation specific value. Configuration is syncing the COTS system with the company process so that system could perform as per the organisation requirement. Configuration is done using the existing structure and functionalities available in the package. Many times calling for change in organisation functioning as per available functionalities of the solution.

Customization: Customization involves developers. If configuration is not supported by default functionalities then with the help of coding in the given compatible language, new features are added to meet the business requirement. These changes are normally outlined on what’s called a functional specification, which is usually written by the responsible functional consultant. Customization can be costly and can complicate future upgrades to the software because the code changes/modifications may not easily migrate to the new version. 

Few Questions before Decision

It is highly improbable to fit any commercially off-the-shelf ERP solution to an organisation or business directly since each entity has its own practices perfected over a long time, operational mechanisms which have worked for the business and local policies operating in the domain. So before taking a leap into investing into ERP implementation, it is extremely important to sit back and think about few questions:
  • Is operation automation the aim or the organisation requires to capture business intelligence for overall growth?
  • Many times organisations/businesses mistake operational automation for an ERP implementation?
  • Which domains need to be addressed and the level of integration between various domains of business/organisation is to be defined?
  • Is any COTS solution fit the scenario? Or totally customised solution is required. If COTS is to be customised more that 20%, it is better to look for alternatives.
  • Is the organisation/business adaptable enough to changes and is it ready to adopt the industry best practices offered by the various ERP solutions?
  • How much investment can be made in the ERP implementation?
  • What is the Return on Investment?
  • What is the goal of ERP solution so that the implementation shall optimise the structure and operations towards business returns? To define the overall goal a top-down approach is to be adopted.
  • What is the timeline and approach to implementation? This is so that the phased manner of implementation can be adopted. In defining the phases the logical bottom up methods are to be adopted for successful implementation.
  • How much integration with legacy systems? Or what is the plan of replacing the legacy systems and data migration?
  • Is it possible to standardise the procedures across various arms of the organisation?
  • What will be the structure of flow of information across various hierarchies in the organisation/business?
  • What is the maintenance cost year on year and what will be the upgrade strategy?
  • What is the expected security level to be built-in?

Building customised hierarchies, often changing reporting structures, dynamic workflows require ERP solution to be fully configurable (or require customisation) with respect to these aspects of implementation. Most of the ERP solution providers have proprietary frameworks for customisation. This is also a business model in which the cost of changes and maintenance is intended to be unreasonably high due to proprietary technology. The expertise is not easily available for the proprietary solutions.


Many times the decision makers confuse office automation with ERP implementation. They would like to automate certain activities like billing and voucher preparation but do not have a clear cut aim of which processes to be converted to automated workflows. Business process automation is an intangible knowledge which needs to be translated into the workflows with authorities and adequate security. There is no direct visibility of the parameters required to be converted to reports and which output reports can drive the input parameters. Translating business acumen to an IT system is an art which depends upon the technical knowledge of ERP solutions as well as the business practices. Sometimes analysing the rationale behind a particular practice can allow the solution designer to tweak with it and bringing an optimum solution in terms of implementing the policy in a different form and following the best practice of the solution. This reduces the customisation effort and the functional requirements of the organisation are also met. But this seldom happens and mutual blame complicates the solutions.

Practically all solution providers like SAP, Oracle, Microsoft, Ramco, Baan and many more COTS offer the above functional domains and modules but require respective proprietary configuration and customisation to fit into any organisation or a business. The advantage of ERP solutions being direct availability of Industry best practices, built in reports, industry standard business intelligence algorithms, integrated frameworks for workflows, rules, interface screens and built in security and authentication features. Most of these OEMs have been strong in one technical domain but over a period of mergers and purchase they acquire other technologies. Thus complete solutions are not integrated by design. Internally modules are joined by various patches to talk to each other. Not all technical domains are strong and have state-of-the art features. 

Employee Self Services (& Custom Developed Solutions)

Employee self-service (ESS) is a term applied to web-based or kiosk based applications that provide employees access to their personal records and payroll details. This is a common requirement of each and every employer being essential for private as well as public employers. This is the first step towards e-governance in which government-to-employees or business employer-to-employees charter is addressed by IT solutions. The features allow employees to change their own contact details, family members & banking information. It also allows administrative tasks like applying for a leave, reviewing of timesheet, inquiring about available loan programs, requesting for overtime payment, and viewing of compensation history, internal transfers, travel approvals, medical benefit requests and submitting of reimbursement slips. With the emergence of ESS, employees are able to transact with their Human Resource office without physical appearance which is considered irrelevant in some transactions. ESS may be operated on an employer's intranet, via a web service or specific kiosks.

ESS is an increasingly prevalent trend in human resources management that allows an employee to handle many job-related tasks that otherwise would have fallen to management or administrative staff. ESS software is available as a stand-alone product or as a component of some larger application, such as an enterprise resource planning (ERP) product. These applications are also built on workflow based systems so that transactions can be stored. ESS is part of a larger movement towards automated and, increasingly, Web-enabled services that encompasses e-support, e-voting, Web self-service, e-procurement, and e-outsourcing, among other possibilities. Each one being a custom solution for the particular domain addressing the specific needs built over the practiced rules of operation. Integrating these solutions for an organisation goes by the design premises of the solutions. The experience of domain experts and IT experts goes into building three tiered solutions (database layer, business layer & GUI layer) which can be scaled and integrated.

Open source technologies offer a plethora of solutions like workflow engines, rules engines, security frameworks, GUI constructs, authentication frameworks and non-proprietary development environments to build a highly concise yet scalable applications. There is strong support available from open source community on these technologies. This approach of implementing specific services through web-portals is one of the easy options to start automation of the organisation/business. Using non-proprietary technologies and integrating open source solutions sometimes can give optimum cost solutions for specifically customised services of a particular domain. It is possible that integrating various domains as an after-thought can be challenge. But well-structured domain by domain implementation can form a great backbone for any kind of large implementations. A detailed look at feasible approaches and available technologies can bring in slow and steady development in all domains. As such this meagre investment can form a strong baseline for bigger and larger integrated implementations as and when an organisation is ready for the uniform enterprise solutions and cultural change. 

e-Governance

Implementing information technology advancements in government setups is first a thought experiment and then an implementation. Government setups are not necessarily profit driven. There is large difference between governance and resource management for profitability. The two being named as e-Governance and ERP respectively. Industry has matured with respect to its business tasks and has evolved many industry practices. Governance practices are yet to mature and form a baseline standards. The primary goal of governments being research, growth of the country and welfare of the citizens. There is clear demarcation between the profit-oriented government sectors and enablement sectors. Thus applying ERP solutions which are based on industry practices and are tuned for profitability are generally misfits in enablement governance oriented sectors. This leads to high rates of failures during implementation.

Almost every sector of operation in a country is governed by various arms of ministries, making government a very wide and varied kind of structure. Though operationally clear in individual setup, it is very complex in unison. Though governed by uniform constitution and rules, interpretations and applications vary widely from domain to domain. There is no standardisation. Due to lack of standardisation across various government arms, no considerable effort has been put by any solution provider to tune the government practices into an ERP solution. The willingness to invest in IT solutions for e-governance by various government arms is different at different levels.

A DoD report states that there are vast dissimilarities between the Defense Department and private sector and it is an "invalid premise" to believe that commercial off-the-shelf ERPs have the capability to meet all DoD needs, even though the software be customized and business processes alerted to conform with the ERP.

The existence of processes without analogue in the private sector leads the government agencies to integrate ERPs with legacy systems. But that undermines the fundamental value proposition of an ERP in the first place--the full integration of processes.

The various departments and arms of government do not have (or do not intend to have) real federation across multiple ERP implementations. In reality, transactions aren't contained within a single organization and thus many functionalities remain out of the scope of IT system.

The leader driven functional domain controls hinder the attempts to standardise the data. The aims and goals of an organisation may differ from others. The rule and policies vary with vested powers. Thus the framework is never standardised. Even the processes are non-standard largely dependent upon the governing authorities. Each changing incumbent un-doing the predecessor’s policies in order to make a mark (a sorry state but true). Frequent changes take its toll on the IT solutions and integration tends to break down

IT implementations require organizations to change from within. Organization's willingness to change behaviour and processes is a prerequisite. There are many within the organisation who do not want to accept this change. Manipulative vested interests de-rail these implementations. The obvious method of de-railing being highlighting the minor flaws in the system. Another factor for failure being lack of support from the domain in-charge whose intangible knowledge needs to be captured into the system. The fear of losing the power of processes and data is all prevalent. The idea of domain experts working towards creating better policies with the help of large database and advanced tools is yet to trickle into the administrators.

IT project owners lack authority to achieve business process changes. Ultimate control being the financial management domain creates a weakness. Leaders with authority and accountability who can ensure necessary amount of change and enforce agreement between disagreeing parties would require to own the implementations. Moreover, they would have to be personally engaged in the process, since delegating their authority will dilute the implementation and eventually fail.

Many features of standard COTS solutions are irrelevant for government sectors like valuation and depreciation. These are built for private sector as part of a tax strategy, which have minimal operational value to the government structure. A diligent accountability based approach (cost accounting, technical goals accountability, administrative accountability & governance accountability) would be better for implementing an IT solution in the government sectors. For this the well thought out practices and efficient mechanisms need to be instantiated through the IT solutions.

Most of the attempts in government have been individual efforts and limited to a small arm of the organisation. Scalable and generic solutions with organisation wide impact have not been attempted. This kind of attempt would have resulted in a framework for a COTS module for government.

Decision to implement ERP, ESS or customised solutions for e-governance, e-support, e-procurement etc is a careful mix of the extent of IT penetration (domain identification), choice of technology, investment cost and cultural change readiness. Implementation should be a task driven by authority and backed by cultural change.

Following reference reports from the internet are a good read. Some parts are used in the above given text.
  • Systemic problems with DoD ERP strategy and implementation, warns report By David Perera, September 23, 2012
  • U.S. Air Force Blows $1 Billion on Failed ERP Project by Robert N. Charette, 15 Nov 2012
  • US Department of Defense – U.S. Air Force, Feb 11th, 2013, Entry from the record in the “Catalogue of Catastrophe” – a list of failed and troubled projects from around the word
  • www.nasa.gov/offices/ocio/portfolio/business.html
  • http://searchsap.techtarget.com
  • http://www.osc.state.ny.us
  • http://sapoverview.blogspot.com