JSPservlet
A PageBox implementation - Diskless version
Support support@pagebox.net
TABLE OF CONTENT 1 Foreword * 2 Architecture and principles * 3 ICP messages * 3.1 Standard * 3.2 Messages 3.2.1 ICP_OP_INI * 3.2.2 ICP_OP_ARCHIVE_STATUS * 3.2.3 ICP_OP_ARCHIVE_UPDATE * 3.2.4 ICP_OP_STAT_QUERY * 3.2.5 ICP_OP_STAT_STATUS * 3.2.6 ICP_OP_STAT _UPDATE * 3.2.7 ICP_OP_LOG * 3.2.8 ICP_OP_FIN * 3.3 Protocol * 3.3.1 Poll mechanism * 3.3.2 Periodic refresh * 4 Parameters * 4.1 cachePath * 4.2 toTrace * 4.3 toStat * 4.4 ID * 4.5 logfile * 4.6 remoteLocations * 4.7 expiration * 4.8 contextPath * 4.9 useBundleSpace * 4.10 Diskless * 4.11 Multicast * 4.12 MulticastGroup * 4.13 MulticastPort * 4.14 RemAddrs * 4.15 RemPorts * 4.16 allPermissionPolicy * 4.17 defaultPolicy * 4.18 keystore * 4.19 keystorePassword * 4.20 CAURL * 4.21 CRLURL * 4.22 CALDAPuser * 4.23 CALDAPpasswd * 4.24 CRLLDAPuser * 4.25 CRLLDAPpasswd * 4.26 CRLperiod * 5 Statistics and troubleshooting * 6 Notes on implementation * 6.1 Dynamic update * 6.1.1 Definition section * 6.1.2 Loaded/defined section * 6.2 Resource handling * 7 Special cases * 7.1 RequestDispatcher * 7.2 Use of SSL * 7.3 Load test * 7.4 Functional tests * 7.4.1 Update * 7.4.2 Security * 7.4.3 Misc * 8 Limitations * 9 Differences between Application Server, JES 2 and diskless versions * 10 Miscellaneous * 10.1 License * 10.2 Deliveries *
TABLE OF FIGURES Figure 1: Diskless architecture * Figure 2: Statistics using ServletStat * Figure 3: log using ServletLog * Figure 4: Selection of the client on DisklessLog * Figure 5: Diskless client log display with DisklessLog * Figure 6: ServletUpdate definition * Figure 7: ServletUpdate list * |
This document describes the diskless version of JSPservletPkg. JSPservletPkg is a complete implementation for servlets and JSP handling from a remote repository and with dynamic update. The diskless version is based on the embedded version and like embedded version has been tested so far only with Java Embedded Server 2 (JES 2) from Sun. The document doesn’t refer to the Embedded version. Diskless version can be configured either as a diskless client that can ROMed and doesn’t need access to a local drive or either as a diskless server that is: Figure 1: Diskless architecture Pagebox instances remain independent: Client data size is small but possibly logs. As diskless clients also send log messages on the wire, it has a consequence. Whereas intensive logging has a negligible impact for Application Server and Embedded server version it should be avoided as much as possible on diskless clients. Diskless clients and server communicate using new ICP messages, so in UDP either in unicast or multicast. The protocol is designed for fault tolerance. Diskless version uses ICP messages as defined in RFC 2186 and RFC 2187. They have a header described by the following C structure: struct _icp_common_t { unsigned char opcode; /* opcode */ unsigned char version; /* version number */ unsigned short length; /* total length (bytes) */ u_num32 reqnum; /* req number (req'd for UDP) */ u_num32 flags; u_num32 pad; u_num32 shostid; /* sender host id */ /* diskless extension */ u_num64 lastModified; /* timestamp */ char[] ID; /* issuer unique identifier * variable length delimited by a 0 and followed by the payload */ }; The opcode describes the message. Diskless version uses opcodes undefined in the standard. It doesn’t use flags and pad fields and sets the version to two. Opcode: 5. PageBox initialization message issued by diskless client. Payload: none. Purpose: Query for archive names. Opcode: 6. PageBox response returned by diskless server when it receives an ICP_OP_INI. Payload: serialized properties whose keys are archive names and values are download locations. Purpose: Provide the list of archives to manage to the diskless clients. Opcode: 13. PageBox archive update message sent by diskless client. Payload: serialized properties whose keys are archive names and values are download locations. Purpose: notify diskless servers updated list of archives managed by the client. This message is sent periodically (every SCANPERIOD = 1 minute in the current version) to address cases where a diskless server is not up when the diskless client adds, delete or change an archive. Opcode: 7. Query for statistics issued by diskless client. Payload: none. The ID is made of the unique identifier of the client followed by slash ("/") and by the archive name. The initialization protocol is: Client Server ICP_OP_INI ==> <== ICP_OP_ARCHIVE_STATUS For each archive managed by the client ICP_OP_STAT_QUERY ==> <== ICP_OP_STAT_STATUS Opcode: 12. PageBox response returned by diskless server when it receives an ICP_OP_STAT_QUERY. Payload: serialized properties whose keys are servlet names and values are invocation numbers. Purpose: Provide current statistics to the client before first invocation by browser users. Opcode: 14. PageBox statistics update message sent by diskless client. Payload: serialized properties whose keys are servlet names and values are invocation numbers. Purpose: Save statistics on server side. This message is sent periodically (every SCANPERIOD = 1 minute in the current version). Opcode: 8. PageBox log message sent by diskless client. Payload: message sent by diskless client. Purpose: Update log on server side. Opcode: 9. Message issued by diskless client to notify diskless servers of its termination. Payload: none. When a diskless server receives ICP_OP_FIN, it closes the relevant log file (identified by the ID carried in ICP_OP_FIN. The protocol is designed for fault tolerance. The unavailability of a server at a given time doesn’t compromise the operations of a set of diskless servers and clients. Poll mechanismAt initialization, a diskless client sends an ICP_OP_INI message to all servers. All up servers answer and the client selects the ICP_OP_ARCHIVE_STATUS answer whose lastModified field is the most recent to get the most up to date archive list. If a server was not up when the client updated lastly its archive definition, its response is not used. The same mechanism is used for statistics (messages ICP_OP_STAT_QUERY and ICP_OP_STAT_STATUS). Suppose the administrator adds an archive to the archive list of a diskless client. Then the client sends an ICP_OP_ARCHIVE_UPDATE message. Server A is up and records the update. Then it is stopped. A server B is started. Now the administrator changes the location of an archive of a diskless client. The client sends an ICP_OP_ARCHIVE_UPDATE. Server B records that update. Now neither server A nor server B has an information that describes correctly the state of the client. To address this issue, update messages are send periodically even if no update occurs in a given session in order to refresh servers with a longer inactivity. It has also an important consequence. One can add a diskless server dynamically. One minute later, it will be up to date. This version is designed to be deployed in JES2 bundles. So its archive must have a Manifest like this: Bundle-Name: JSPservlet Bundle-Description: JSPservlet Bundle-Vendor: Alexis Grandemange (GNU GPL 2). Bundle-Version: 1.0.1 Bundle-DocURL: http://java.sun.com/products/embeddedserver Bundle-ContactAddress: alexis.grandemange@pagebox.net Bundle-Activator: JSPservletPkg.JSPhandler Import-Package: org.osgi.service.http, javax.servlet; specification-version=2.1.1, javax.servlet.http; specification-version=2.1.1, com.sun.jes.service.http.auth.basic Import-Service: com.sun.jes.service.http.auth.basic.BasicSchemeHandler There is no limitation to the number of bundles, which can be configured, in a given Application Server. Its configuration is specified by a property file named Bundle-Name.properties where Bundle-Name is the name you gave to the bundle in the manifest. Here is an example of configuration: cachePath=/jdj toTrace=TRUE toStat=TRUE logfile=/jdj/log.txt remoteLocations=/jdj/jdj.properties allPermissionPolicy=C:/temp/JES2/allPermission.policy defaultPolicy=C:/temp/JES2/default.policy keystore=C:/temp/JES2/keystore keystorePassword=keystorePswd adminUser=admin adminPasswd=admin ID=Hamlet contextPath=/jdj useBundleSpace=TRUE Diskless=FALSE MulticastGroup=228.10.10.10 MulticastPort=9089 Multicast=TRUE CRLURL=ldap://localhost/CN=alexis,CN=agrandem,CN=CDP,CN=Public Key Services,CN=Services,CN=Configuration,DC=alexis CAURL=ldap://localhost/CN=alexis,CN=AIA,CN=Public Key Services,CN=Services,CN=Configuration,DC=alexis CRLLDAPuser=Alexis/Users/Administrator CRLLDAPpasswd=gunnm007 CALDAPuser=Alexis/Users/Administrator CALDAPpasswd=gunnm007 expiration=30 CRLperiod=30 This file acts as the web.xml of Application Servers’ version. I bolded the parameters that doesn’t exist in web.xml version. The package must contain: WEB-INF web.xml // as the file above classes JSPupdate.class JSPservletPkg CRLchecker.class DisklessLog.class ICPhandler$DisklessCltHandler.class ICPhandler$IcpHeader.class ICPhandler$Log.class ICPhandler$LogPrivilegedAction.class ICPhandler.class JSPhandler$ClassEntry$ServletInfo.class JSPhandler$ClassEntry$Stat.class JSPhandler$ClassEntry.class JSPhandler$Log.class JSPhandler$Scanner.class JSPhandler.class JSPloader$ClassInfo.class JSPloader$ProtectionDomainInfo.class JSPloader$ResourceEntry.class JSPloader.class JSPloaderException.class JSPresourceServlet.class ResourcePrivilegedAction.class PageBoxAPI.class ServletLog.class ServletStat.class ServletUpdate.class We detail below JSPservlet.properties parameters. cachePathcachePath is the location where jars are locally stored after been retrieved from remote location. Default value: C:/temp. Tells if the tool must write diagnostic messages. Default value: false. Tells if the tool must record statistics. Default value: false. If true, statistics are recorded per archive in a file cachePath/archive.stat, which is a property file, for instance: #Sun Nov 19 23:20:32 CET 2000 TestServlet/FileAccess=3 helloImg=8 TestServlet/OtherServlet=2 SnoopJSP=3 TestServlet/ForwardingServlet=4 SnoopServlet=2 Allows the deployer to specify a unique identifier. PageBoxAPI allows retrieving this ID. Application: deployment of a large number of instances. No default value. Tells where the tool must write diagnostic messages. Default value: $CachePath/log.txt. Location of a property file containing jar names and associated URLs. Default value: $CachePath/$ContextPath.properties where ContextPath indicated the name the war file is deployed with. JSPupdate updates this file. It is OK to modify it manually but don’t expect to retrieve your comments. You can set this parameter to minimize the round trip number between the browser and the server. JSPservlet sets the Expire header field of static content (content with an extension different of class). It computes the Expire as current_time + expiration. expiration unit is second. Default value: 5 seconds. contextPath is the equivalent of the name you give to the JSPservletPkg Web Archive in Application Servers configuration. Consider you configure JES2 http server to handle requests toward http://myserver:8080, you will invoke a servlet defined in archive myarchive.jar in mypath/myservlet with http://myserver:8080/contextPath/myarchive/mypath/myservlet.Default: none. Must be specified. Boolean. If true, uses JES 2 BundleContext’s getDataFile to create File objects in the persistent storage area provided for the bundle by the framework.
I recommend setting it to true first because it is a requirement for well-behaving bundles and second because it provides useful features such as removing all bundle files when the bundle is uninstalled. If you set it to true, path parameters are relative to the bundle space root but:
The reason is these files must be installed separately and are not related to a bundle. Default: true. Boolean. If true, starts as a diskless client and doesn’t write on local file system. In this case, you don’t need to set parameters such as logfile and cachePath. If false, starts as a diskless server and writes on local file system for it and its clients.
Default: false. Boolean. If true, uses Multicast. In this case, it uses the multicast group specified by MulticastGroup and the port specified by MulticastPort, both for listening and sending. If Multicast=false, uses Unicast. In this case, listens on the address specified by MulticastGroup and the port specified by MulticastPort. A diskless client must be configured with two other parameters:
Default: true. Specify a valid multicast group with a string xxx.xxx.xxx.xxx, for instance 228.10.10.10 if Multicast=true. Otherwise specifies an IP address or name. Default: none. Mandatory. Specify an IP port. Default: none. Mandatory. Contains the IP addresses of diskless servers as strings separated by commas, for instance Miami, Orlando, Lauderdale. Contains the IP ports of diskless servers as strings separated by commas, for instance 9001, 9000, 9002. In this example, the diskless client will send messages to Miami:9001, Orlando:9000, Lauderdale: 9002. If it is set, allPermissionPolicy is the path to a policy file with syntax conforming to the Java 2 security specification. If defaultPolicy is also set it means:
The following no-brainer allPermissionPolicy will work in all cases: grant { permission java.security.AllPermission; }; Note that you should never grant permissions in cachePath/archive policy as these files are downloaded from the archive location. Default: the parameter has no default value. If it is set, defaultPolicy is the path to a policy file with syntax conforming to the Java 2 security specification. If allPermissionPolicy is also set it means:
If no policy applies to the archive, then defaultPolicy is used The Java server itself will run with the permission described in allPermissionPolicy. Default: the parameter has no default value. keystore is the name of the key store in cachePath directory, for instance "keystore" but not "/mydir/keystore" or "mydir/keystore". If keystore is set, when JSPservletPkg downloads an archive, it tries
The archive user has no longer to administrate security. It is appropriate for trusted providers. Default: the parameter has no default value. Password JSPservletPkg uses to access the keystore. Default: the parameter has no default value. When JSPservletPkg implements sandboxes, it processes signed archives and retrieve classes certificate chain. If CAURL is set, it connects to this URL and expects to retrieve a Certificate Authority certificate used in the classes certificate chain. If it fails to connect to the CA or if what it retrieves is not a certificate, it logs an ERROR entry with "directory access failure". If the certificate is valid but not present in a class certificate chain, it invalidates the class just as if one of its certificates was revoked. Note that JSPservletPkg doesn’t load the class and therefore doesn’t raise a security exception but a class not found exception. If you set this parameter, you MUST add JNDI to your Java Server CLASSPATH in JDK 1.2. In JDK 1.3, you don’t have to, as JNDI is included in JDK. Default: the parameter has no default value. When JSPservletPkg implements sandboxes, it processes signed archives and retrieve classes certificates. If CRLURL is set, it connects to this URL and expects to retrieve a Certificate Revocation List (CRL) used to check if a class certificate is revoked. If it fails to connect to the CA or if what it retrieves is not a CRL, it logs an ERROR entry with "directory access failure". If the CRL is valid and it finds one of the class certificates in it, it invalidates the class. Note JSPservletPkg doesn’t load the class and therefore doesn’t raise a security exception but a class not found exception. If you set this parameter, you MUST add JNDI to your Java Server CLASSPATH in JDK 1.2. In JDK 1.3, you don’t have to, as JNDI is included in JDK. Default: the parameter has no default value. Principal used to connect to the Directory server to retrieve CAURL. Default: the parameter has no default value. If it is not set, the tool connects to the Directory server without credential and password (LDAPpasswd). Password used to connect to the Directory server to retrieve CAURL. Default: the parameter has no default value. Principal used to connect to the Directory server to retrieve CAURL. Default: the parameter has no default value. If it is not set, the tool connects to the Directory server without credential and password (LDAPpasswd). Password used to connect to the Directory server to retrieve CAURL. Default: the parameter has no default value. Defines how often the tool will connect to check for CRL updates in seconds. Default: 7 * 24 * 3600 (1 week). Figure 2: Statistics using ServletStat Figure 3: log using ServletLog You can use the log path I list at the top of the screen to find out the bundle location. However it is not a hidden secret: It starts with the value of com.sun.jes.framework.bundles.baseurl, which is D:\Documents and Settings\Administrator\jescache in this case, followed by bundlebundle_id\data, where bundle_id is a number set by the framework. ServletLog reads the log file. Log entries not being in memory, it can be used only for diskless servers. To display diskless client logs, you must use DisklessLog and use the URL http://serverURL/DisklessLog. Note that if you have two or more diskless servers on the same subnet, you can choose any of them.Then you are prompted to select the appropriate diskless client: Figure 4: Selection of the client on DisklessLog You select the client and click on List. Then you get the same display as with ServletLog: Figure 5: Diskless client log display with DisklessLog You see the path of the log and clear it. Dynamic update of jar file is supported though ServletUpdate. ServletUpdate display is split in two sections:
ServletUpdate uses GET mode. It is a servlet defined in the JSPservlet package. You must fill the JAR Name field without extension. If you don’t fill the remote location the current location is reused. It is the JAR file URL minus the file name. Assuming you specified a jar name myjar and a remote location http://www.mydownloadsite.com, JSPserletPkg will download http://www.mydownloadsite.com/myjar.jar and persist your action in RemoteLocations with a property myjar=http://www.mydownloadsite.com.Note the URL of ServletUpdate http://localhost:6080/jdj/JSPupdate. jdj is the contextPath. If you deploy different JSPservlet bundles, you must select the appropriate JSPservlet to require an update. When you updates, the local cache is removed and the jar is always loaded from the remote location. You can also use ServletUpdate to add a jar.Figure 6: ServletUpdate definition It supports a trace checkbox and two buttons:
You can safely require an update when pages are accessed. GC allows to invoke System.gc() in the Java server context. Typical case of that is image handling. Suppose a JSP whose URL is http://www.mysite.com/jdj/myjar/myJSP references an image with a relative path images/myimage.gif. The Application Server looks for http://www.mysite.com/jdj/myjar/images/myimage.gif and therefore invokes JSPresourceServlet.JSPresourceServlet looks first for images/myimage.gif in the archive. If it doesn’t find it, it retrieves the download location in the myjar’s RemoteLocations property. If it finds its remote location is www.mydownloadsite.com, it downloads the image from www.mydownloadsite.com/images/myimage.gif.The only case where you must modify your code is when you need using a RequestDispatcher either to include or forward a request. You can use JSPservlet.getJAR() helper function. RequestDispatcher rd = getServletContext().getRequestDispatcher( JSPservlet.getJAR(getClass().getClassLoader()) + OtherServlet); The reason is as a servlet developer you should not hardcode neither the Web Application name as the standard enforces it, neither the jar name where your servlet will be deployed. JSPservlet allows using SSL to download archives using the ssl.jar bundle. The explanation below is not JSPservlet related but it still can help you to configure the thing. As it can fail at your first attempt, I recommend you reading the JSSE API user guide. It explains you how to trace the SSL connection with javax.net.debug=all and gives you a lot of information. You need first to SSL enable the Web Server where you download the archive from. It implies generating a key pair. The server keeps the private key internally and asks you to submit a certificate request to the Certificate Authority of your choice. During the SSL handshaking, jsse will check it knows the certificate chain the Web Server presents it using the keystore specified by com.sun.jes.impl.keystore.store. A default keystore is defined in JES installation path/lib/tlscerts. tlscerts format is jks, which means you can use keytool to display or update its content. If the server Certificate Authority certificate is not in the list, you have to add it. Retrieving it depends on the browser and on the Certificate Authority. You can ask for a DER encoded or for a base 64 format. Then you can add your server certificate with: keytool -import -file C:\TEMP \serverCA.cer -alias myCA –keystore <JES2-home>/lib/tlscerts -storepass passphrase With up to 3000 servlets in a single jar file. The performance impact of tracing is below 10%. The sandbox performance impact is not measurable. ServletUpdate Multiple archives, multiple web applications Invalid archive name/location Non-signed archive
Signed archive with and without sandbox. With sandbox:
1 policy file per web application or 1 per archive. No permission granted to a signed archive. Policy/certificate download I checked the tool supports:
Limitations can be:
I chose using Sun packages as much as possible.
public class TagextActivator implements BundleActivator { public void start(BundleContext bundlecontext) {} public void stop(BundleContext bundlecontext) {} public TagextActivator() {} }
I packaged the bundle with this manifest: Manifest-Version: 1.0 Bundle-Vendor: apache.org Bundle-Version: 1.0 Bundle-Activator: tagext.TagextActivator Bundle-DocURL: http://jakarta.apache.org Created-By: 1.2.2 (Sun Microsystems Inc.) Bundle-Name: tagext Bundle-ContactAddress: agrandemange@pagebox.net Export-Package: javax.servlet.jsp.tagext Bundle-Description: tag extension Import-Package: javax.servlet,javax.servlet.jsp,javax.servlet.http,org.apache.jasper,org.apache.jasper.runtime You can consider an alternate solution where you add the JSP related stuff to JSPservlet. JSP packaging of JES 2 will probably be enhanced in the coming months and the most important point here is that embedded servers can support JSP V 1.1. Application Server version supports Servlet 2.2. JES 2 version supports JES 2 Servlet 2.1. Diskless version is a superset of JES 2 support with a support for diskless – ROM able – clients. All versions should follow closely framework versions in the future. The major design difference is between JES/diskless and Application Server version:
This has two important implications:
I chose to design the JES 2 version in that way to minimize its footprint. Diskless version is close to JES 2 version because differences are hidden in
This code makes however a difference in term of footprint – 40% bigger. I maintain JES 2 version mainly for that reason. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 of the License. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. Package name: JSPLoaderPkg. Source files: CRLchecker.java JNDI/LDAP code DisklessLog.java Servlet to display diskless client logs on diskless server side ICPhandler.java ICP message handler JSPloader.java class loader JSPloaderException.java package exception JSPhandler.java Web application handler JSPresourceServlet.java Resource handling servlet PageBoxAPI.java API to get the PageBox ID and log user messages ResourcePrivilegedAction.java resource loader ServletLog.java servlet to clear and display log ServletStat.java servlet to display stats ServletUpdate.java update servlet Documentation:
|