Legal knowledge-based system improvement through open-source development

Jaap J. Dijkstra and Mathieu H. Paapst
Section Law & ICT, Faculty of Law, University of Groningen
j.j.dijkstra@rug.nl, mpaapst@home.nl

Abstract. This paper discusses the possibilities of open-source legal knowledge-based system (LKBS) development for public administration. Developing legal knowledge bases in an open-source community of lawyers, lobby groups and other interest groups could bring forth more accurate legal knowledge models. The content of the knowledge base could even become subject of legal and political dispute, ensuring a more direct interaction between the LKBS’s knowledge model and legal practice. Also Franken’s principles of proper use of ICT - especially accessibility, integrity, transparency and flexibility - could be utilized better in an open-source community. However, getting lawyers involved with open-source knowledge-base development may prove to be a difficult task.

1. Introduction
In public administration we can see an increasing use of information and communication technology. Government agencies have started to use legal knowledge-based systems (LKBSs) and other computerized decision-making tools to handle complex administrative tasks ([1]). The use of automated decision making tools has become an efficient and cost-saving practice in public administration. Moreover, computerized decision-making tools can help to ensure an equal treatment by the government before the law. So, gradually artificial intelligent computer support, like the use of LKBSs, is gaining ground in e-government.

However, the use of LKBSs is not without difficulties. Several studies have shown that LKBSs used in legal practice can and do make mistakes (see for instance [1][2][3]). To ensure a correct use of LKBSs Franken ([4]) identified six general principles of proper use of ICT within public administration: accessibility, confidentiality, integrity, authenticity, flexibility and transparency. According to Franken these principles should be met when government organizations use computer support for legal decision-making.

The principles of confidentiality and authenticity deal with the information that has to be processed. Confidentiality should ensure that the information is only available to those who are authorized to have access to the information, and the principle of authenticity deals with the question whether the information can be trusted. These principles do not deal with the LKBS itself, but with the information that is processed in an individual legal case. The principles of accessibility, integrity, flexibility and transparency can be applied to the LKBS and the knowledge base itself. This paper focuses on the question whether open-source LKBS development could help to improve the application of Franken’s principles. First we will take a closer look at the principles of accessibility, integrity, flexibility and transparency. Then we will discuss some of the ideas behind open-source software and see whether open-source development could lead to a better application of the principles of correct ICT use in government. We conclude the paper with discussing some organizational and legal issues for putting open-source LKBS development into practice.

2. Accessibility, integrity, transparency and flexibility of LKBSs

2.1. The principle of accessibility
The principle of accessibility of LKBSs is similar to the more general principle of ‘equality of arms’ in legal practice. All parties involved in a legal dispute should be given access to the legal information and knowledge that has been used to come to a decision. This principle is in line with a more general principle of proper administration: the justification principle. An administrative body should give the foundation for its decision (article 3:46 and 3:47 of the Dutch General Administrative Law Act). According to Groothuis ([1]) this would mean that if the decision is made by a LKBS, the reasoning behind the automated decision should be explained. This requirement can be satisfied by giving the applicant or his or her legal advisor access to the LKBS, given that the LKBS is transparent to the user in its presentation of its knowledge rules. In an evaluation study of the Dutch General Administrative Law Act, Bergfeld, Kaspersen and Lodder ([5]) recommend that government should provide public access to government databases and information systems when there are no privacy issues involved. Public access to decision support systems would be in line with this advice. Indeed, in 2002 the Dutch government decided to give open access to a computerized support system (the BOS system) used by the public prosecutor for reasons of equality of arms and transparency of government ([6]).

2.2. The principle of integrity
The principle of integrity deals with the quality of decision making by the LKBS. The quality of the decisions made by the LKBS depends heavily on the quality of the underlying knowledge ([7]). When a knowledge base does not contain all relevant legal rules, the LKBS can be misleading, causing people to ignore critical aspects of a problem or perhaps form misguided conclusions ([8][9]). Sometimes lawyers have different opinions about the law, lawyers can change their minds and the law itself can even change over time. In short, the knowledge model of a LKBS is often incomplete and sometimes incorrect. Indeed, in a field study concerning a LKBS in administrative law, Groothuis and Svensson ([3]) found that the system made mistakes due to its limited scope and depth of the knowledge model. De Bruin, Prakken and Svensson ([2]) identify ‘partial support’ as an important cause of incorrect LKBS use. The user does not see the limits of the computer support, and therefore expects too much from the LKBS. In her Ph.D. study Groothuis ([1]) concludes that LKBSs do not guarantee juridical correct decisions. Government agencies that decide to use LKBSs should take additional measures to maintain control over the quality of the decisions in fields that are not, or insufficiently, covered by these systems.

2.3. The principle of transparency
The principle of transparency says that the working of the LKBS should be transparent for the user of the system. A civil servant should understand the decision-making model that has been applied by the LKBS he or she is using. At least he or she should be able to verify the reasoning applied in a specific advice by the LKBS. A LKBS is not an infallible oracle. Consequently, the inferences of a LKBS should be treated as advice, and there is the implicit assumption that the user will judge the applicability of the advice ([10]). To meet this requirement LKBSs are equipped with explanation facilities to help the user to understand the reasoning behind a LKBS advice. The user can evaluate the inference process by examining the rules, the facts and the conclusions used to generate the advice. However, even though most LKBSs provide explanation facilities, the actual transparency for the user can be questioned. Dijkstra ([11]) found that in laboratory experiments users of LKBSs hardly use the explanation facilities provided. Users tend to trust the advice without thoroughly examining it. Petty and Cacioppo ([12]) argue that people may adopt attitudes on grounds other than their understanding and evaluation of an argumentation behind a persuasive message. When users of LKBSs are hardly motivated, or do not have the knowledge or experience to scrutinize the argumentation made by a LKBS, they are likely to accept the advice without studying it because they see the LKBS as a trustworthy adviser. So, explanation facilities do not guarantee that the reasoning behind a LKBS advice is transparent for the user.

2.4. The principle of flexibility
The principle of flexibility addresses the problem of changes in the law. Whenever the law changes, a LKBS that has been written to apply this part of the law has to be updated as well. The LKBS should be flexible enough to make these changes possible within reasonable time.

Several authors have addressed the problems mentioned above. Especially the problems of accessibility and integrity have been given much attention in the literature on legal computer support. For instance, Snellen ([13]) argues that software used in decision making by government agencies should be publicly available. Groothuis ([1]) recommends that government agencies should take additional measures to prevent errors, especially when the system only offers limited support. De Vey Mestdagh, Dijkstra and Oskamp ([14]) think that an authorized seal of approval for LKBSs used for e-government is needed to prevent the use of inadequate LKBSs.

However, one of the reasons that LKBSs are often not publicly available or validated is because these systems are developed and maintained by commercial software companies. These software companies are the copyright holders of the LKBSs. The government agencies have only limited copyrights licenses and do not have the right to give public access to the systems. Moreover, the agencies do not have access to the source code of the LKBSs themselves. Therefore, accessibility, full transparency, integrity validation and flexibility are obscured because of the software companies’ copyrights.

3. Open-source software development
In software development the open-source concept has been introduced to challenge the copyright monopoly of software companies and to improve the quality of software. Open-source software gives the user full access to the source code to ensure the computer program’s accessibility, transparency and integrity. The general idea behind open-source software is that public access to the source code should lead to better software development. Raymond ([15]) sees decentralized peer review, lower costs and improved quality of software as the main advantage of open-source software development. He compares traditional industrial software development with the building of a cathedral where ‘everything is carefully crafted by individual wizards or small bands of mages working in splendid isolation’, whereas open-source software development can be seen as ‘a great babbling bazaar of different agendas and approaches’. Linux has proven that this bazaar style of software development can work, and can result in good software development. Co-development with a group of software engineers and peer users is a good approach to rapid code improvement and effective debugging. All in all, this should result in better quality software. In traditional software development user feedback and response to user feedback is a problem in the development process. In open-source development, given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix will be obvious to someone. Linus (Torvalds)’s Law states: “Given enough eyeballs, all bugs are shallow” ([15], p. 30).

Even though the first open-source projects have been private initiatives, several software companies (e.g. Netscape) have adopted this new form of software development. Open-source software development does not exclude commercial exploitation. For instance, one could think of selling services instead of closed software. This is what Red Hat does with the open-source operating system Linux.

4. Open-source LKBS development
To meet the principles of proper use of LKBSs in government, open-source development of legal knowledge bases could prove to be an interesting new way of implementing computerized decision support in legal practice. The open-source (or open-content) development of the knowledge base should ensure the validity of the knowledge model from the perspective of the LKBS users. Even though the knowledge-based system shell could also be developed in an open-source context, this paper focuses on the open-source development of the content of the LKBS: the legal knowledge model. When interest groups in a legal domain would have full access to the knowledge base, more opinions and interpretations can be evaluated in the construction of the knowledge base, resulting in a more balanced – or at least more transparent – knowledge model.

4.1. Accessibility through open source
First of all, the principle of accessibility is fully met when all stakeholders have full access to the knowledge base and the underlying legal knowledge model. The advantage is that one can monitor: 1) if the laws have been properly modelled into the knowledge base; 2) if certain interpretations of the law modelled differ from one’s own opinion; and 3) if certain new developments in the law (jurisprudence) have already been modelled into the knowledge base. Moreover, everyone concerned could give suggestions for improvements of the knowledge model, and could warn if there are clear bugs in the knowledge base.

4.2. Integrity through open source
The accessibility of the knowledge base should ensure a better quality of the knowledge model. Similar to the bazaar model of open-source software development, the knowledge model of a LKBS can be criticized and improved by all interest groups concerned. Different points of view should lead to improvements in the knowledge base, or at least it should become clear when the government agency and for instance a big law firm or an interest group have dissenting opinions. One could even develop a knowledge base that enables the modelling of dissenting opinions ([16]). Open access to legal knowledge sources (e.g. codified law and jurisprudence) is a general principle to ensure public access to the law and public involvement in what should be the law. With full access to the rule base of a LKBS, its knowledge model could even become the subject of legal dispute in court. A court could decide if the law has been correctly modelled in a LKBS. With closed source software LKBSs it is very difficult for a court to criticize a LKBS.

4.3. Transparency through open source
Accessibility of the knowledge base seems to imply transparency as well. However, we think that the transparency of the knowledge base is the most difficult challenge for successful open-source LKBS development. There is a huge difference between giving lawyers full access to a rule base and their actual ability and motivation to scrutinize the knowledge rules. Dijkstra ([17]) found in several experiments that users of a LKBS, even though they had full access to the content of the knowledge-base, hardly used this option to verify the knowledge model. Lawyers do not seem to be very motivated to study for instance production rules and maybe they lack the ability as well. Without getting the legal specialists involved, open-source LKBS development will not work.

Getting legal specialists involved implies that these jurists have to see the LKBS not merely as a tool for efficient decision making, but as a subject of legal discussion. It should be clear to them that they can benefit from studying a knowledge base. According to Davis’s Technology Acceptance Model ([18]), the perceived usefulness of a software product is crucial for its actual use. Likewise one could expect that the lawyers’ perceived usefulness to scrutinize and validate the knowledge base in an open-source LKBS setting is crucial for the success of open-source LKBS development. If the lawyers are not (intrinsically) motivated to study the knowledge base and to contribute to the knowledge base, then the project will fail.

Besides perceived usefulness, the Technology Acceptance Model identifies perceived ease of use as another indicator for software use. Inspecting the LKBS has to be made easy for legal specialist. Therefore, the way the knowledge is represented should make it for legal specialists as easy as possible to study the knowledge model. Van Engers, van Driel and Boekenoogen ([19]) have addressed this problem. They compared several forms of legal knowledge representations for learnability and usability. They found that jurists preferred the use of decision tables, but this resulted in lower overall performance in applying the knowledge. The use of production rules and decision trees improved test subjects’ performance but they liked these forms of knowledge representation far less. The use of scenarios (a kind of decision tree) led to the best performance. However, at the time of the experiment the program the authors used could not automatically generate these scenario representations.

4.4. Flexibility through open source
Experience from open-source software projects teaches that these projects can be very flexible when it comes to bug reporting, fixing bugs and software innovation. According to Raymond ([15]) it is crucial for the open-source development model that the users have access to and insight in the source code. Bug report or other comments by non-source aware users normally tend not to be very useful. An underlying mismatch between the tester’s and the developer’s mental model frustrates a fruitful cooperation. Open-source development makes it far easier to develop a shared representation ground. When comments on the quality of a knowledge base are directly linked to the ‘source code’ – specific rules in the rule base – the time needed by the developers to fix the problem will decrease. Indeed, the mismatch between the mental conception of a software tool’s functionality between lawyers and programmers is a well-known problem in software development for legal practice ([19]). Therefore, open-source development, where both developers and users refer to the same rule base, could lead to more flexible LKBSs.

We can conclude that the application of the open-source concept to LKBSs development could lead to a better fulfilment of Franken’s principles of proper ICT-use in government. The next question is how open-source LKBS development could be put into practice. We will discuss some organizational and legal questions concerning this matter.

5. Business models for open-source LKBS development
One of the lessons to be learned from open-source software development is that the open-source concept does not have to exclude commercial software development. Analogue to commercial Linux distributors, a commercial LKBS distributor could ensure the compatibility of the LKBS with the database environment of a government agency.

Commercial software developers are probably needed for successful open-source LKBS development. Usually an open-source product will not be programmed from the group up in an open-source community. Somebody has to take the lead to originate the project, build a first version of the software product, and start with community-building. With open-source LKBSs projects the expert system shell needs to be an existing product. Probably this will be a standard commercial product that is part of a database suit used by government agencies. In the Netherlands Centric has developed such an expert system shell that can be used with their information systems. It would come in handy if there is a freeware ‘light’ version of the shell available to be used by lawyers outside the agencies to access and evaluate the open-source rule bases. Furthermore, some special knowledge representation tools could help to make the content of the knowledge bases as transparent as possible for the legal specialists (like for instance the scenarios used by van Engers et. al. [19] or the Legal Knowledge Interchange Format that will be developed in the European Estrella project (see the Estrella project description on http://www.jur.uva.nl/leibniz/)).

The organisation of an open-source community usually consists of four layers ([20]). The first layer consists of people who just use the software. They incidentally may report a bug they find. With the open-source development of an LKBS one would for instance find civil servants and legal advisers in this layer. The second layer would consist of LKBS users who actively notify the rest of the community when they encounter a bug. These would be super users and legal specialists from the government agencies and legal specialist from universities, law firms and interest groups. The ‘bug’ reports may even result from legal disputes in court about the content of the knowledge base. The third layer includes people who have implemented updates for the knowledge base. These would be lawyers with some technical background who enjoy translating legal knowledge into production rules. At the fourth level are the people who actively develop the knowledge base. They also have to decide which improvements are to be implemented in a new authorised version of the knowledge base. The people in the fourth layer could be employees of the software company or the central government agency.

6. Legal implications of open-source LKBS development
From a legal perspective there is a difference between open-source software and open-source knowledge bases. The legal concept behind open-source software is that copyright law is used to ensure the open-source character of the software developed. Because copyright law states that software can never be used without permission of the copyright holder, one can restrict the conditions under which the software use is allowed. Therefore, the open-source character of a software product can be ensured in the licence agreement. If one accepts the licence agreement one also accepts the open-source conditions. If one does not accept the licence agreement, the use of the software is not allowed because of the copyright infringement. Everyone who joins the open-source development community has to accept that his or her contributions will be subject to these same open-source conditions.

It is unclear if production rules should be treated as source code. One could argue that an expert system shell is a kind of interpreter (similar to for instance a Java-interpreter) and that the content of the rule base is a set of instructions similar to a computer program. If so, then open-source licences can be used to ensure the open-source character of the knowledge base. However, if a knowledge base should not be treated as software, an open-content licence should be used to ensure the open-content character. If a knowledge base is considered to be a set of rules (data) then database protection law could be applicable as well. If so, the user licence should also deal with bringing database protection under the open-content regime ([21]).

7. Conclusions
Open-source development of LKBSs might prove to be an interesting approach to get legal specialists involved in the construction and validation of LKBSs used in e-government. The use of the open-source concept would also contribute to a more transparent government. In future government agencies might even be forced to grant open access to the knowledge models in the LKBSs they use to handle their administrative tasks. Open-source LKBS development would also improve proper ICT use according to Franken’s principles, especially the principles of accessibility, integrity, transparency and flexibility. However, the problem of transparency will not be solved by just giving open access to the knowledge base. More research is needed to study how knowledge from a LKBS can be represented to make it easier for legal specialists to validate the knowledge and to contribute to the knowledge base in an open-source community.

References
[1]	Groothuis, M.M. (2004). Beschikken en digitaliseren: Over normering van de elektronische overheid. Ph.D. dissertation University of Leiden.

[2]	Bruin, H. de, Prakken, H. and Svensson, J. (2002). The Use of Legal Knowledge-Based Systems in Public Administration: What Can Go Wrong? Proceeding of the Fifteenth Annual Jurix Conference (eds. Bench-Capon, T.J.M., Daskalopulu, A. and Winkels, R.G.F.), pp. 123-132.

[3]	Groothuis, M.M. and Svensson, J.S. (2000). Expert system support and juridical quality. Proceeding of the Thirteenth Annual Jurix Conference (eds. Breuker, J., Leenes, R. and Winkels, R.), pp. 1-10.

[4]	Franken, H. (1993). Kanttekeningen bij het automatiseren van beschikkingen. In: Beschikken en automatiseren: Preadviezen voor de Vereniging van Administratief Recht (eds. Franken, H., Snellen, I.Th.M., Smit, J. and Venstra, A.W.), pp. 11-49, Alphen aan den Rijn: Samson H.D. Tjeenk Willink.

[5]	Bergfeld, J.P.R., Kaspersen, H.W.K. and Lodder, A.R. (2000). Wob & ICT: Onderzoek naar de gevolgen van toepassing van Informatie- en Communicatietechnologie voor de Wet openbaarheid van bestuur. Amsterdam: Vrije Universiteit.

[6]	Lodder, A.R. and Oskamp, A. (2004). ICT-toepassingen in het strafrecht. In: Strafrecht & ICT (ed. Koops, B.J.), pp. 215-237, Den Haag: SDU.

[7]	Spreeuwenberg, S., Engers, T. van and Gerrits, R. (2001). The Role of Verification in Improving the Quality of Legal Decision-Making. Proceeding of the Fourteenth Annual Jurix Conference (eds. Verheij, B. Lodder, A.R., Loui, R.P., and Muntjewerff, A.J.) pp. 1-15.

[8]	Norman, D.A. (1993). Things that make us smart: Defending human attributes in the age of the machine. Reading, MA: Addison-Wesley.

[9]	Smith, B.C. (1995). The limits of correctness in computers. In: Computers, ethics & social values (eds. Johnson, D.G. and Nissenbaum, H.), pp. 456-469, Englewood Cliffs, NJ: Prentice-Hall.

[10]	Svensson, J.S. (1997). Tessec: Improving the quality of administrative decision making under the General Assistance Act. The new review of applied expert systems, Volume 3, pp. 143-154.

[11]	Dijkstra, J.J. (2000). User interaction with legal knowledge-based systems. Proceeding of the Thirteenth Annual Jurix Conference (eds. Breuker, J., Leenes, R. and Winkels, R.), pp. 11-21.

[12]	Petty, R. E., and Cacioppo, J. T. (1986). Communication and persuasion, central and peripheral routes to attitude change. New York: Springer Verlag.

[13]	Snellen, I.Th.M. (2004). Naar een verbeterde ICT-huishouding van de overheid, In: Trias Informatica: Verslag van het symposium, Den Haag: Raad voor het Openbaar Bestuur. (http://www.rob-rfv.nl)

[14]	Vey Mestdagh, C.N.J. de, Dijkstra, J.J. and Oskamp, A. (2002). Kwaliteitsbewaking van juridische informatie- en kennistechnologie. In: Informatietechnologie voor juristen (eds. Oskamp, A. and Lodder, A.R.), pp. 137-163, Deventer: Kluwer.

[15]	Raymond, E.S. (2001). The cathedral and the bazaar, Sebastopol: O’Reilly.

[16]	Vey Mestdagh, C.N.J. de (1997). Juridische Kennissystemen, Rekentuig of rekenmeester? Het onderbrengen van juridische kennis in een expertsysteem voor het milieuvergunningenrecht, Ph.D. dissertation University of Groningen.

[17]	Dijkstra, J. J. (1998). On the use of computerised decision aids: An investigation into the expert system as persuasive communicator. Ph.D. dissertation University of Groningen.

[18]	Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Management Information Systems Quarterly, Volume 13, pp. 319-340.

[19]	Engers, T. van, Driel, L. van, and Boekenoogen, M (2002). The Effect of Formal Representation Formats on the Quality of Legal Decision-Making. Proceeding of the Fifteenth Annual Jurix Conference (eds. Bench-Capon, T.J.M., Daskalopulu, A. and Winkels, R.G.F.), pp. 63-71.

[20]	Wendel de Joode, R. van, Bruijn, J.A. de and Eeten, M.J.G. van (2003). Protecting the Virtual Commons, Den Haag: T.M.C. Asser.

[21]	Paapst, M.H. (2005). Openbaarheid door open source: Over de toepassing van open source concepten op juridische kennissystemen bij de overheid, Master’s thesis University of Groningen.