Version 0.2 N. Soriano MSX community May 9, 2007 MSX-UNAPI: Unified procedure for API definition and discovery on MSX computers Abstract This document describes MSX-UNAPI, a standard procedure for defining, discovering and using new APIs (Application Program Interfaces) for MSX computers. The goal is to provide a unified way to design and use BIOS-like software for the various kinds of new hardware that is being developed for these computers, altough it can be used for software-only solutions as well. Table of Contents 1. Introduction 1.1. Motivation 1.2. Goals 1.3. Non-goals 1.4. Glossary 1.5. The big picture 2. API specification rules 2.1. Rule 1: The API identifier and version number 2.2. Rule 2: One single entry point on Z80 page 1 or 3 2.3. Rule 3: Z80 registers usage 2.4. Rule 4: Routine number ranges 2.5. Rule 5: The API information routine + the implementation name and version 2.6. Rule 6: Implement code for the API discovery procedure 2.7. Rule 7: Install the RAM helper 3. API implementations discovery procedure 3.1. The MSX extended BIOS 3.2. Steps of the discovery procedure 3.3. How to implement support for the discovery procedure 4. The RAM helper Appendix A. Acknowledgements Appendix B. Document version history Author's Address 1. Introduction 1.1. Motivation MSX was presented in 1983 as a standard for home computers, in the form of a set of specifications to be followed by manufacturers. Any brand was able to make its own model of computer; as long as the machine met the MSX standard, it was considered an MSX computer. This allowed for flawless interoperation between computers of different makers, something not as common at the time as it is nowadays. An important part of the MSX specification was the BIOS, a set of routines that -amongst other things- provided a standarized way to access the harwdare resources of the machine. This provided certain degree of freedom to manufacturers, which could -to some extent- freely choose the hardware parts that would build up their given model of MSX computer. Time passed and the MSX standard was discontinued by all manufacturers long time ago. However, many people still using their old MSX computers, and some of them even develop new hardware for these machines. Hard disk interfaces, sound cards and network cards are examples of hardware that has been developed for MSX computers past the commercial live of the standard. Altough the appearance of new hardware is of course a good thing, this causes a problem for software developers. Not having any authority dictating specifications for the MSX standard anymore, each hardware developer freely choses the way their hardware will have to be accessed from software; in other words, there are no specifications to follow at the time of designing the API (Application Program Interface) for the new harwdare. As a result, hardware extensions performing the same functions but being developed by different persons are software-incompatible (unless developers privately agree on using compatible APIs, which is not always possible). The intent of this document is to propose a solution for this problem: a standard, unified procedure for defining, discovering and using new APIs for MSX computers, aimed especially (but not exclusively) to the hardware development field. 1.2. Goals The main goals of this specification are: o To provide a minimum set of rules to be followed by APIs that adhere to this specification. These rules are to be taken in account when designing (and later implementing) the various APIs. They are very simple, impose only a few restrictions on compliant code, and in practice allow any kind of API software to be written. o To provide a standard procedure for discovering and using APIs that adhere to this standard and are installed on the MSX computer running the UNAPI aware software. This procedure is based on the MSX extended BIOS mechanism, and is very easy to implement and even easier to use. 1.3. Non-goals It is NOT the intent of this specification, and/or is outside the scope of this document: o To impose the described specification to developers or users. Altough the authors hope that the MSX-UNAPI specification will be useful for the MSX community, this document is merely a proposal, and adhering to it is of course optional. o To arbiter who should desing the APIs. Common sense suggests that APIs should be designed by the first person or team that develops a given type of hardware (or software), but the decision of such thing is outside the scope of this document. o To describe any API other than for illustration purposes. This documents just tells a procedure for designing APIs; these must be described in sepparate documents. 1.4. Glossary Before proceeding any further, a glossary of terms used along this document will be introduced. It will be useful to understand the rules and procedures described later. MSX-UNAPI Short for "MSX unified API definition and discovery standard". A set of rules to be followed when designing and implementing APIs, aimed mainly (but not exclusively) to hardware developers; and a procedure for discovering any such APIs available to application software. API specification API stands for "Application Program Interface". The description of a set of routines that perform some action (which usually, but not necessarily, involve accessing some kind of hardware), including the precise description of input and output parameters, as well as the side effects caused by the execution of each routine. UNAPI compliant API specification An API specification that conforms to the rules for API specifications described in this document. API specification identifier An alphanumeric, case-insensitive string made of up to 15 characters, which uniquely identifies an UNAPI compliant API specification. For example the identifier of an API specification for Ethernet cards could be "ETHERNET". See Section 2.1 for more details. The XXXXX UNAPI specification The specification for an UNAPI compliant API whose specification identifier is XXXXX. For example, "the Ethernet UNAPI specification". XXXXX UNAPI compliant API implementation Real code that builds up a set of routines which conforms to an UNAPI compliant API specification (and more precisely, to the XXXXX UNAPI specification). There may be several implementations of one single specification, each made by a different person or team; but if they strictly conform to the API specification, they are indistinguishable with respect to usage and behavior. API implementation name An alphanumeric string made of up to 64 characters, which uniquely identifies an API implementation. For example the name of an implementation of the Ethernet UNAPI could be "ObsoNET", while the name of other implementation could be "UltraNET Supercard". See Section 2.5 for more details. XXXXX UNAPI client software Application software that invokes the routines of an XXXXX UNAPI compliant API implementation. For example, a TCP/IP stack could act as client software of the Ethernet UNAPI. The client software must use the discovery procedure (see Section 3) to locate the available API implementations, and usually it does not matter which one is used if more than one is found. Through this document, and unless otherwise stated, the term "API specification" will actually have the meaning of "UNAPI compliant API specification". The same applies to references to API implementations. 1.5. The big picture This section provides an fictional scenary that briefly shows the steps that are needed in order to design and implement UNAPI compliant APIs. The goal is to provide the reader a basic understanding about how all the whole thing works, so that it is easier to read the detailed rules and procedures that are provided in the remaining of the document. Suppose that a user named H.G. Wells builds a time machine for MSX computers. He wants to make the software to control it to be UNAPI compliant. To achieve this, since it is the first time machine ever developed for MSX, he first must design the API specification. These are the steps he follows for it: 1. Chooses a short name for the API specification. He chooses "TIME_MACHINE". (see Section 2.1) 2. Designs the routines that will compose the API. He designs three routines: travel backwards, travel forwards, and return to original time. These routines will have routine numbers 1 through 3. (see Section 2.2, Section 2.3 and Section 2.4) Now he must write a real implementation of the API, to be included in ROM together with the time-travel hardware. He does then the following: 1. Chooses a name for the API implementation: "Well's Time Machine BIOS" (see Section 2.5) 2. Implements the actual code for the three API routines (again, see Section 2.2, Section 2.3 and Section 2.4) 3. Implements the API discovery procedure, so that when the time machine is present in a MSX machine, it can be discovered by using extended BIOS (see Section 3) The device turns out to sell quite well in MSX users meetings, and some users develop software that uses it, from multi-epoch games to virtual history books. All of this software uses the UNAPI discovery procedure (see Section 3) in order to find out at run time whether the time machine is present or not. Soon afterwards, another user with hardware design knowledge, Dr. Brown, develops another kind of time machine for MSX. Since he wants the software that already exists for Well's hardware to be compatible with his own device, he develops the software so that it is compatible with the TIME_MACHINE API specification, by following the same steps that Wells did. There are, however, some important differences with the Well's machine: 1. The API implementation name is now "Brown's flux-capacited time machine". 2. The device is connected to MSX via serial port, so it has no ROM and the API implementation is therefore installed in mapped RAM. This implies that when this API is installed, it installs the RAM helper. (see Section 4) 3. There is an extra routine which is needed to calibrate the flux capacitor. This routine is implemented as an implementation- specific routine (see Section 2.4), and has the routine number 128. Now that there are two different time travel devices around, an user named M. McFly wants to develop a windowed graphic interface to control the time machines. It develops the software to discover, and interact with, TIME_MACHINE APIs; therefore it works with both kinds of devices. Moreover, as an extra feature, it checks the implementation name of the API implementation it uses (see Section 2.1), and when it is the Brown's implementation, the application offers extra functionality to calibrate the flux capacitor. 2. API specification rules This section describes the rules that must follow any UNAPI compliant API specification, as well as their implementations. 2.1. Rule 1: The API identifier and version number An API specification must have an alphanumeric identifier composed of up to 15 characters. Allowed characters are letters (having an ASCII code below 128), digits, and the following ones: - _ / . ( ) The API identifier is case-insensitive, so for example, "ETHERNET" is the same as "Ethernet". An API specification must have a version number composed of main version number in the range 1-255, and secondary version number in the range 0-255. Specifications with higher version numbers must be backwards compatible with older versions. The first released specification document should have version 1.0. Versions 0.x are allowed as pre-releases, with no guarantee of backwards compatibility between them. Digits may be used in the identifier when they are meaningful to describe the API (for example "WIFI_IEEE802.3"), but they must NOT be used to indicate the specification version. 2.2. Rule 2: One single entry point on Z80 page 1 or 3 There must be one single entry point for the API routines. This entry point must be accessible in any address on one of these places: o A ROM or non-mapped RAM slot, on Z80 page 1 (address between #4000 and #7FFF). o A mapped RAM segment, provided that it will be connected to Z80 page 1 (address between #4000 and #7FFF) when called. o System RAM on page 3 (address above #C000). It is outside the scope of this specification to define how RAM memory (mapped RAM segments and space in system RAM on page 3) is allocated. If necessary, API routines may return a pointer to hard-coded information to be read by the client software (as the mandatory API information routine does with the API implementation name, see Section 2.5). In this case, the information must be in the same place as the API implementation itself. That is, in the same ROM slot (in Z80 page 1), in the same RAM segment (if this is the case, a page 1 address is returned), or in system RAM on page 3. This way it is easy to obtain the name by using inter-slot or inter-segment (see Section 4) reads. As an exception to this rule, API implementations residing in page 1 (ROM or a mapped RAM segment) may return an address in system RAM on page 3. Therefore the client software must be always prepared to either perform inter-slot or inter-segment reads, or direct memory reads, in order to retrieve the data. If necessary, API routines may require the client software to specify a buffer where dynamic data will be read from, or written to (for example, packets to be sent to or retrieved from the network in the case of an Ethernet API). In this case, the API specification may impose a restriction to the client software, so that it must NOT specify any page 1 address as the buffer address. 2.3. Rule 3: Z80 registers usage Z80 registers are used by the API routines in the following way: o Register A is used at input to specify the routine to be executed. Each routine has associated a number in the range 0-254; how these numbers are allocated is described in Section 2.4. It can be used also as an output parameter, otherwise it is corrupted after the routine is executed. o Registers F, BC, DE and HL may be used freely as input and/our output parameters. Registers not used to hold output parameters are corrupted after the routine is executed, unless otherwise stated in the routine definition. o Registers IX, IY must not be used for input parameters, to allow for inter-slot and inter-segment calls. They can be used as output parameters, otherwise they are corrupted after the routine is executed. "The registers are corrupted after the routine is executed" means that the routine code may freely make internal use of the registers, and it does not need to restore their original values before finishing. 2.4. Rule 4: Routine number ranges As stated before, each API routine has associated a number which must be passed in register A when calling the API entry point. The numeric range 0-255 is divided in four ranges of routines: o 0: This value corresponds to the API information routine. It is a mandatory routine that must be present on all implementations of any UNAPI compliant API. The parameters and behavior of this routine are described in Section 2.5. o 1-127: Specification routines. The behavior of these routines is defined in the appropriate API specification document. o 128-254: Implementation-specific routines. API implementation developers may freely use this range of routines to offer additional capabilities not present in the original API specification; if this is done, these routines must be described in the documentation of the implementation. For example, one implementation of the Ethernet UNAPI could use this range of routines to offer WiFi connectivity, while other could offer basic internetworking capability. Note that different implementations may use the same numbers for different routines, always inside this range. o 255: This value is reserved for a possible future extension mechanism. Numbers must be assigned to routines in increasing order, starting with 1 for specification routines and starting with 128 for implementation-specific routines; it is not allowed to leave holes in the function number ranges. If client software invokes a non- existing routine (that is, calls the API implementation entry point passing in A a number not assigned to any routine), nothing must happen and the code must return with AF, BC, DE and HL unmodified. 2.5. Rule 5: The API information routine + the implementation name and version Every API implementation, no matter which API specification is implementing, must mandatorily implement the API information routine. This routine has always the routine number 0 and has the following input and output parameters: Input: A = 0 Output: HL = Address of the implementation name string DE = API specification version supported. D=primary, E=secondary. BC = API implementation version. B=primary, C=secondary. The implementation name string must be zero terminated and composed of up to 64 printable characters. The name may be any descriptive text, but must NOT contain any version information, and is case- sensitive. The implementation name must be placed in the same place as the API implementation itself (see Section 2.2). Usually, client software will use the first suitable API implementation it founds, and will use the implementation name merely to show information to the user. However, if the client software is able to use the implementation-specific features of any given implementation, it must check the implementation name to know whether the desired implementation-specific routines are available or not. Client software should check the API specification version supported by the implementation, since newer implementation versions may have routines not available in older versions. Rules for the implementation version are similar to rules for the specification version (see Section 2.1): first release version should be 1.0, and 0.x versions are allowed for pre-releases. However, the backwards compatibility rule now applies to implementation-specific routines only, since the compatibility with the specification routines is indicated by the "API specification version supported" parameter. It is not allowed for an API implementation to go backwards in the API specification version supported when the API implementation version is increased. For example, if implementation version 2.0 supports specification version 1.5, it is illegal for implementation version 2.1 to claim compliance with the specification version 1.4; it must support specification version 1.5 or higher. Otherwise, backwards compatibility between API implementations could not be assured. 2.6. Rule 6: Implement code for the API discovery procedure Every API implementation, no matter which API specification is implementing, must mandatorily support the API discovery procedure. See Section 3. 2.7. Rule 7: Install the RAM helper Every API implementation that is installed on a mapped RAM segment must, at installation time, install the RAM helper unless it is already installed. See Section 4. 3. API implementations discovery procedure Section 2 described the rules for designing UNAPI compliant APIs. These API specifications are used by developers to make API implementations, which are composed of real code that once installed on an MSX computer (whatever the installation method is), becomes available to client software. This section describes the API implementations discovery procedure, that is, the steps that client software must follow to find out how many implementations of a given API specification are available at run time, and to gather information about each implementation (in which slot/segment are placed, where the entry point is, and the name and version information), as well as how API implementations can include support for this procedure. 3.1. The MSX extended BIOS The discovery procedure is based on the MSX extended BIOS, which is a mechanism available on MSX computers that allows standard BIOS to be extended with new routines. This section explains the general rules of the extended BIOS mechanism, and later sections explain how this is used for the UNAPI discovery protocol. Extended BIOS provides a five byte hook named EXTBIO at address #FFCA. To check whether this hook has been initialized with any value or is uninitialized, look at bit 0 of the byte at address HOKVLD (#FB20). A value of one for this bit means that the hook contains a valid jump instruction, previously set by system code or user code. The EXTBIO hook contents may be replaced with a jump instruction to custom code, provided that the old hook contents are saved somewhere. If the hook is not initialized, it should first be filled with five RET instructions, and bit 0 of HOKVLD should be set; from this point, consider it as being initialized. The EXTBIO hook is called with a code named "device identifier" in register D, a "function identifier" in register E, and input parameters in AF, BC and HL. The code called in this way must look at the device identifier to check if the request is for itself. If not, it must jump to the old hook (the one saved at installation time) with AF, BC, DE and HL unmodified. When the device identifier matches the one expected by the code, it performs the action requested (as per the function identifier) and either returns parameters in, or corrupts, AF, BC and HL; DE is always preserved. IX, IY and the alternate registers are always corrupted, no matter whether any suitable code for the specified device identifier is executed or not. This procedure allows to chain together multiple BIOS extensions, each having its own device identifier. These identifiers were once assigned by MSX manufacurers, and later on, users have developed software that patch extended BIOS, freely choosing their own identifiers. 3.2. Steps of the discovery procedure Client software willing to discover how many implementations of a certain API are available at run time must perform these steps: 1. Copy the zero-terminated API specification identifier to address #F847. This is a 16 byte buffer named ARG, used normally by Math-Pack (mathematic routines package of MSX BIOS). 2. Set Z80 registers as follows: A=0, B=0, DE=#2222. 3. Call the EXTBIO hook. 4. When EXTBIO returns, B contains the number of installed implementations of the specified API. DE is preserved and AF, C, HL are corrupted. After the previous steps have been performed, and provided that at least one implementation is installed (B>0), do the following to gather information about a given implementation: 1. Make sure that the API specification identifier is still at ARG. 2. Set Z80 registers as follows: A=implementation index (from 1 to the number of available implementations), DE=#2222. 3. Call the EXTBIO hook. 4. When EXTBIO returns, Z80 registers contain the following information about the specified implementation: A = Slot where the implementation code is placed B = RAM segment where the implementation code is placed (#FF if not in mapped RAM) HL = Routines entry point address (if a page 3 address, A and B are meaningless) DE is preserved. F and C are corrupted. With this information, now you know how to call the implementation entry point in order to invoke the API routines: o If the entry point address is a page 3 address, ignore A and B and perform direct calls to that address. o Otherwise, if B=#FF, use inter-slot calls (for example by using the BIOS routine CALSLT) to invoke the routines. o Otherwise, use inter-segment calls (you can use the RAM helper for this, see Section 4) to invoke the routines. Usually, the first thing to do at this point is to call the API implementation information routine (see Section 2.5) to obtain the implementation name and version (this is not mandatory, though). From here, client software can invoke the API routines as needed. 3.3. How to implement support for the discovery procedure Section 3.2 provided a functional description of the API implementations discovery procedure. For this procedure to work, all implementations of an UNAPI compliant API must include code to support this procedure. This section explains how this code should be implemented. First, when the API implementation is installed (or at system boot, for ROM based code), the existing EXTBIO hook must be backed up (if EXTBIO hook is not initialized, initialize it first as explained in Section 3.1). Then, put a jump instruction or an inter-slot or inter-segment call pointing to your EXTBIO manager code. Second, include in the API implementation code to manage EXTBIO calls (that's where the new EXTBIO hook points to). This code must perform the following steps: 1. Check that register pair DE holds the value #2222. If not, jump to the old EXTBIO hook with AF, BC, DE, HL unmodified. 2. If A=#FF, jump to the old EXTBIO hook with AF, BC, DE, HL unmodified (this is to allow the RAM helper to be installed, see Section 4). 3. Check that the string placed in ARG matches the identifier of the API specification that your code implements. If not, jump to the old EXTBIO hook with AF, BC, DE, HL unmodified. Remember that the strings comparison must be done in a case-insensitive way. 4. If A=0, increase the value of register B by one, and jump to the old EXTBIO hook with A and DE unmodified. 5. Otherwise, if A=1, put the appropriate information about the implementation in registers A, B, and HL, as described in Section 3.2, and return (do NOT call the old EXTBIO hook) with DE unmodified. 6. Otherwise, decrease A by one, and jump to the old EXTBIO hook with DE unmodified. Note that given the way the discovery procedure is designed, the implementations installed in the first place have assigned the highest implementation index numbers. Usually this has no impact on the design on client software but it is a good thing to know it. 4. The RAM helper Once an API implementation has been discovered, it is easy for client software to invoke the API routines if the API entry point is located at page 3 (using direct calls) or in a ROM slot (using inter-slot calls, for example via the BIOS function CALSLT). However, for API implementations installed in a mapped RAM segment, it is not so easy to invoke the API entry point. MSX-DOS 2 provides system routines to execute code placed on an arbitrary segment, but the client code must still ensure that the appropriate RAM slot is switched on page 1. For MSX-DOS 1 it is even worse, since no mapper support routines are provided. To solve this issue, and in order to ease the development of client software, this specification includes the concept of the RAM helper. The RAM helper consists of two routines that are placed at system RAM on page 3, each having an entry point in the same area. One of the routines allows to easily perform inter-segment calls, while the other allows to perform inter-segment data reads. To check for the presence of the RAM helper, and to obtain the address of its two routines, EXTBIO must be called with DE=#2222, HL=0, and A=#FF. If the RAM helper is not installed, then HL=0 at output; otherwise HL will contain a page 3 address of a jump table containing the following two entries: +0: Call a routine in a mapped RAM segment Input: IYh = Slot number IYl = Segment number IX = Target routine address (must be a page 1 address) AF, BC, DE, HL = Parameters for the target routine Output: AF, BC, DE, HL, IX, IY = Parameters returned from the target routine +3: Read a byte from a RAM segment Input: A = Slot number B = Segment number HL = Address to be read from (higher two bits will be ignored) Output: A = Data readed from the specified address F, BC, DE, HL, IX, IY preserved The installation code for every API implementation that is to be installed on a mapped RAM segment must check if the RAM helper is installed in the system (by using the procedure based on a EXTBIO call described above). If not, the implementation installer itself must both install the RAM helper routines, and appropriately patch the EXTBIO hook so that the RAM helper can be discovered by using the explained procedure. How page 3 RAM is allocated in order to install the RAM helper is outside the scope of this specification. Installing the RAM helper is mandatory (for API implementations which reside in mapped RAM), but using it is optional; client software may choose to use its own code for calling the API routines and reading the API data. However, it is recommended to use the RAM helper since it significantly reduces the complexity of client software. Appendix A. Acknowledgements This document was produced using xml2rfc v1.32 (of http://xml.resource.org/) from a source in RFC-2629 XML format. Appendix B. Document version history o Version 0.2 * Added the "The big picture" section (Section 1.5) Author's Address Nestor Soriano MSX community Email: konamiman@konamiman.com URI: http://www.konamiman.com