Monday, November 24, 2008

the Belgian eID and PDF: Forms and Digital Signatures

Source: http://itext.ugent.be/articles/eid-pdf/index.php?page=4#start
===============

the Belgian eID and PDF:
Forms and Digital Signatures

These are my notes for my GovCamp presentation in Brussels (September 21, 2006; between OSCon and DrupalCon).

I was asked to talk about the Belgian eID; the eID is the new Belgian identity card in the form of a smartcard. I am not an eID expert, so I decided to change the focus of the presentation to a technology I know rather well: the Portable Document Format (PDF).

In this article you will learn:

  • how to create a PDF document containing an interactive form (an AcroForm)
  • how to fill this form with data retrieved from the eID
  • how to use your eID to add an ordinary digital signature (recipient signature) as well as a certifying signature (author signature) to a PDF document

The conclusion will be that you really don't need to be an eID expert to achieve all this. If I can do it, so can you ;-)


Summary

In this presentation we have learned more about the following topics:

Forms in a PDF document

We have learned:

  • How to create a form using iText
  • How to use a PDF form in a web application
  • How to use a PDF form as a template
  • How to flatten a form
  • How to retrieve data from a form

Note that XFA forms are not yet supported in iText, and that special usage rights can only be added using official Adobe software.

The Belgian eID and PDF

We have used an eID to:

  1. Fill in a form with data retrieved from the eID
  2. Learn about the different certificates on an eID
  3. Add one or more recipient (or ordinary) signature to a PDF document
  4. Add one author (or certifying) signature to a PDF document

Note that we have use the qualified (or non-repudiation) certificate to add the signatures. The hash of the PDF document content was generated on a Vasco DigiPass 850 smartcard reader.

We didn't go into details. I hope you agree you didn't have to be an eID specialist to understand the principles of using an eID in combination with PDF documents. You can find out more about PDF in my book. At the bottom of this page you'll find some interesting links to pages about the eID.

Extra links

Resources

All examples in this article were created using the Java version of iText. There are also some .NET ports of iText available. If you have .NET eID software, it should be possible to adapt the examples so that they work in a .NET environment too.

Other articles on this subject

Relevant mailinglist questions regarding signatures

  • A digital signatures example to get you started in .NET: GMANE Nabble
  • How to use an image for the signature appearance: GMANE Nabble
  • Appearances created by iText: GMANE
  • How to find the names of the signature fields: GMANE Nabble
  • Can I remove a digital signature? GMANE Nabble
  • How to remove a digital signature? GMANE Nabble
  • How to avoid the removal of a digital signature? GMANE Nabble
  • Multiple signatures: why don't they work? A very interesting extension of this short article, explaining the different levels of Certifying Signatures: GMANE Nabble

Note that some questions can't be answered because some requirements depend on wrong assumptions:

  • You can't change a document without invalidating the signature: GMANE Nabble
  • You can't change the appearance of an existing digital signature: GMANE Nabble
  • You can't set the Usage Rights of a PDF file with iText: GMANE Nabble

About the book

This article was made from the notes of my presentation for GovCamp Brussels. It is an extended version of Appendix D of the book iText in Action.

You will need this book if you want to know more about creating and manipulating PDF documents.

For instance:

About the eID

This is a list of useful links if you want to know more about the eID

  1. GoDot.be: the website of Danny De Cock
  2. eid.belgium.be: Belgium's eID portal
  3. The Belgian Identity Card (Overview): a short introduction to the eID written by Danny De Cock, Christopher Wolf, and Bart Preneel.
  4. FIDIS: a study on id documents
  5. rijksregister.fgov.be: the National Registry
  6. Certipost: check and download certificates
  7. MicroSoft: the eID page
  8. Thesis Alexander Goossens (in Dutch): eID; Wat is het? Hoe werkt het? Wat zijn de mogelijkheden?

Further reading on cryptography and digital signatures

  1. Wikipedia: digital signature, Certificate Authority, PKI,...
  2. The PKI Page
  3. Bouncy Castle: Java cryptography resources and open source code
  4. Legal issues: A Comparison of Digital and Handwritten Signatures (Paper for MIT 6.805/STS085: Ethics and Law on the Electronic Frontier, Fall 1997)
  5. Digital Signatures and Electronic Documents: A Cautionary Tale (IFIP Conference on Communications and Multimedia Security, September 2002)

Acknowledgements

I would like to thank the following people:

  • Paulo Soares wrote the code to provide support for ordinary signatures in iText. He also wrote the article How to sign a PDF using iText
  • Antonio Iacono provided source code to support certifying signatures in iText
  • Danny De Cock wrote the GoDot tool and maintains a site full of useful information on the eID.
  • Philippe Frankinet wrote the first code sample on how to sign a PDF using iText
  • Bart Van Herreweghe for the invitation to present iText at GovCamp Brussels

iText digital signature example

Source: http://article.gmane.org/gmane.comp.java.lib.itext.general/21374
==========================

Dear Danny, Bruno and Paulo
I've also tested iText, the EID card and IAIK (http://jce.iaik.tugraz.at) without any problem.

Here is the code :
package be.nsi.security.signature.pdf;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.security.MessageDigest;
import java.security.cert.Certificate;
import com.lowagie.text.*;
import com.lowagie.text.pdf.*;
import be.godot.sc.engine.BelpicCard;
public class PDFSign
{
String[] args;

public PDFSign(String[] args)
{
super();
this.args = args;
}

public void selfSignMode() throws Throwable
{
File file = new File(args[0]);
if (!file.exists())
throw new FileNotFoundException("File "+ file.getPath() + "doesn't exist");

PdfReader reader = new PdfReader(file.getPath());
FileOutputStream fout = new FileOutputStream(args[0]+".selfsignmode.pdf");
PdfStamper stamper = PdfStamper.createSignature(reader, fout, '\0');
PdfSignatureAppearance sap = stamper.getSignatureAppearance();

Certificate[] certs = new Certificate[1];

BelpicCard scd = new BelpicCard("");
certs[0] = scd.getNonRepudiationCertificate();
//sap.setCrypto(iKeyStore.getPrivateKey(eidCertificate.getAlias(),null), certs, null, PdfSignatureAppearance.WINCER_SIGNED);
sap.setCrypto(null, certs, null, PdfSignatureAppearance.SELF_SIGNED);
sap.setReason("How to use iText with the new belgian electronic identity card");
sap.setLocation("Belgium");
// comment next line to have an invisible signature
sap.setVisibleSignature(new Rectangle(100, 100, 200, 200), 1, null);
sap.setExternalDigest(new byte[128], new byte[20], "RSA");
sap.preClose();

// Self-Sign mode
PdfPKCS7 sig = sap.getSigStandard().getSigner();

byte[] content = streamToByteArray(sap.getRangeStream());
byte[] hash= MessageDigest.getInstance("SHA-1").digest(content);
byte[] signatureBytes = scd.generateNonRepudiationSignature(hash);

sig.setExternalDigest(signatureBytes, null, "RSA");
PdfDictionary dic = new PdfDictionary();
dic.put(PdfName.CONTENTS, new PdfString(sig.getEncodedPKCS1()).setHexWriting(true));
sap.close(dic);
}

public static byte[] streamToByteArray(InputStream stream) throws Throwable
{
if (stream == null)
{
return null;
}
else
{
ByteArrayOutputStream byteArray = new ByteArrayOutputStream();
byte buffer[] = new byte[1024];
int c = 0;
while ( (c = stream.read(buffer)) > 0)
{
byteArray.write(buffer, 0, c);
}
byteArray.flush();
return byteArray.toByteArray();
}
}

public static void main(String[] args)
{
try
{
PDFSign pdfsign = new PDFSign(args);
pdfsign.selfSignMode();
}
catch(Throwable t)
{
t.printStackTrace();
}
}
}

Philippe Frankinet
Technical Analyst (p.frankinet nsi-sa.be)

NSI S.A. (www.nsi-sa.be)
Chaussée de Bruxelles, 174 A
B-4340 Awans - Belgique
Tél. : +32 (0)4 239 91 50 - Fax : +32 (0)4 246 13 08

How to sign a PDF using iText and iTextSharp

Source: http://itextpdf.sourceforge.net/howtosign.html
====================

How to sign a PDF using iText and iTextSharp


Index


Introduction

iText supports visible and invisible signing using the following modes:

  • Self signed (Adobe.PPKLite)
  • VeriSign plug-in (VeriSign.PPKVS)
  • Windows Certificate Security (Adobe.PPKMS)

Signing and verifying with iText is easy, and it's always done the same way, the difficult part comes with the key and certificate generation. This is a quick guide and doesn't replace the know how you should have on cryptography to make the best use of the technology.


How to sign

Signing is done with this simple code:

KeyStore ks = KeyStore.getInstance("pkcs12");
ks.load(new FileInputStream("my_private_key.pfx"), "my_password".toCharArray());
String alias = (String)ks.aliases().nextElement();
PrivateKey key = (PrivateKey)ks.getKey(alias, "my_password".toCharArray());
Certificate[] chain = ks.getCertificateChain(alias);
PdfReader reader = new PdfReader("original.pdf");
FileOutputStream fout = new FileOutputStream("signed.pdf");
PdfStamper stp = PdfStamper.createSignature(reader, fout, '\0');
PdfSignatureAppearance sap = stp.getSignatureAppearance();
sap.setCrypto(key, chain, null, PdfSignatureAppearance.WINCER_SIGNED);
sap.setReason("I'm the author");
sap.setLocation("Lisbon");
// comment next line to have an invisible signature
sap.setVisibleSignature(new Rectangle(100, 100, 200, 200), 1, null);
stp.close();

How to obtain the private key my_private_key.pfx and the certificates my_certificates.p7b will be explained later.


How to certify

In addition to signing the documents can be certified. Any change to the document that goes against the certification level will make it lose the certified status. The certification level can be: NOT_CERTIFIED, CERTIFIED_NO_CHANGES_ALLOWED, CERTIFIED_FORM_FILLING and CERTIFIED_FORM_FILLING_AND_ANNOTATIONS. To certify a document sign it as in the previous example but add

sap.setCertificationLevel(PdfSignatureApperance.CERTIFIED_NO_CHANGES_ALLOWED);
before closing. This implementation of certification will only work in Acrobat 7 and later.

How to do multiple signatures

Signing as just described will invalidate any existing signatures in the document. To add another signature a new revision is needed. To create a new revision just create the PdfStamper the following way and do the rest as already explained:

PdfStamper stp = PdfStamper.createSignature(reader, fout, '\0', null, true);

How to verify

Verifying is a three step process:

  • Was the document changed?
  • What revision does the signature cover? Does the signature cover all the document?
  • Can the signature certificates be verified in your trusted identities store?
Here's an example on how to do it:

KeyStore kall = PdfPKCS7.loadCacertsKeyStore();
PdfReader reader = new PdfReader("my_signed_doc.pdf");
AcroFields af = reader.getAcroFields();
ArrayList names = af.getSignatureNames();
for (int k = 0; k < names.size(); ++k) {
String name = (String)names.get(k);
System.out.println("Signature name: " + name);
System.out.println("Signature covers whole document: " + af.signatureCoversWholeDocument(name));
System.out.println("Document revision: " + af.getRevision(name) + " of " + af.getTotalRevisions());
// Start revision extraction
FileOutputStream out = new FileOutputStream("revision_" + af.getRevision(name) + ".pdf");
byte bb[] = new byte[8192];
InputStream ip = af.extractRevision(name);
int n = 0;
while ((n = ip.read(bb)) > 0)
out.write(bb, 0, n);
out.close();
ip.close();
// End revision extraction
PdfPKCS7 pk = af.verifySignature(name);
Calendar cal = pk.getSignDate();
Certificate pkc[] = pk.getCertificates();
System.out.println("Subject: " + PdfPKCS7.getSubjectFields(pk.getSigningCertificate()));
System.out.println("Document modified: " + !pk.verify());
Object fails[] = PdfPKCS7.verifyCertificates(pkc, kall, null, cal);
if (fails == null)
System.out.println("Certificates verified against the KeyStore");
else
System.out.println("Certificate failed: " + fails[1]);
}

Note that if you were using the self signed mode you would have to load kall with the certificate provided by whoever signed the document. The certificate could be loaded this way:

CertificateFactory cf = CertificateFactory.getInstance("X509");
Collection col = cf.generateCertificates(new FileInputStream("self.p7b"));
KeyStore kall = KeyStore.getInstance(KeyStore.getDefaultType());
kall.load(null, null);
for (Iterator it = col.iterator(); it.hasNext();) {
X509Certificate cert = (X509Certificate)it.next();
kall.setCertificateEntry(cert.getSerialNumber().toString(Character.MAX_RADIX), cert);
}

How to generate the keys for the self signed mode

To generate a key to self sign:

keytool -genkey -keyalg RSA -alias myname -keypass password -keystore keystore.ks -dname "cn=Paulo Soares, c=PT"

Answer the questions and at the end you have a self signed certificate at keystore.ks. You can omit the -dname option but dont't forget to put the 2 letter country code when asked to. You can now sign the document with:

KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
ks.load(new FileInputStream("keystore.ks"), "keystore_password".toCharArray());
String alias = (String)ks.aliases().nextElement();
PrivateKey key = (PrivateKey)ks.getKey(alias, "password".toCharArray());
Certificate[] chain = ks.getCertificateChain(alias);
PdfReader reader = new PdfReader("original.pdf");
FileOutputStream fout = new FileOutputStream("signed.pdf");
PdfStamper stp = PdfStamper.createSignature(reader, fout, '\0');
PdfSignatureAppearance sap = stp.getSignatureAppearance();
sap.setCrypto(key, chain, null, PdfSignatureAppearance.SELF_SIGNED);
sap.setReason("I'm the author");
sap.setLocation("Lisbon");
// comment next line to have an invisible signature
sap.setVisibleSignature(new Rectangle(100, 100, 200, 200), 1, null);
stp.close();

To generate the certificate that can be validated at the receiving end do:

keytool -export -alias myname -file export.cer -keystore keystore.ks

The generated certificate export.cer can be imported by iText or Acrobat for validation.


How to sign with VeriSign and Microsoft Windows Certificate

To sign documents with VeriSign you need a key that must be certified by VeriSign. You can acquire a 60 day trial key or buy one at https://digitalid.verisign.com/client/class1MS.htm.

The Acrobat VeriSign plug-in (that you can get here) will only work with VeriSign certified keys but the Microsoft Windows Certificate will work with any trusted certificate and in addition to the VeriSign certificate you can also use a free Thawte certificate found at https://www.thawte.com/email/index.html.

After going through all the CA instructions you'll end up with a certificate in your IE. You'll have to export the private key including all the certificated in the certification path. The result is a .pfx file with your private key that can be used to sign the documents.


How to acquire a permanent key from a CA

To acquire a permanent key from a CA such as VeriSign you usually need to go through the following steps:

  1. Generate a self signed key with
    keytool -genkey -keyalg RSA
  2. Generate a certificate signing request with
    keytool -certreq
    the result is base64 data that you must cut and paste in the CA site when prompted
  3. The CA will send a certificate reply chain that you must import with
    keytool -import
    using the same alias

How to sign with an external signature and a standard filter

Some examples follow. Replace with your signing code in the appropriate places.

An example with an external hash in Windows Certificate Mode

KeyStore ks = KeyStore.getInstance("pkcs12");
ks.load(new FileInputStream("my_private_key.pfx"), "my_password".toCharArray());
String alias = (String)ks.aliases().nextElement();
PrivateKey key = (PrivateKey)ks.getKey(alias, "my_password".toCharArray());
Certificate[] chain = ks.getCertificateChain(alias);
PdfReader reader = new PdfReader("original.pdf");
FileOutputStream fout = new FileOutputStream("signed.pdf");
PdfStamper stp = PdfStamper.createSignature(reader, fout, '\0');
PdfSignatureAppearance sap = stp.getSignatureAppearance();
sap.setCrypto(key, chain, null, PdfSignatureAppearance.WINCER_SIGNED);
sap.setReason("I'm the author");
sap.setLocation("Lisbon");
// comment next line to have an invisible signature
sap.setVisibleSignature(new Rectangle(100, 100, 200, 200), 1, null);
sap.setExternalDigest(null, new byte[20], null);
sap.preClose();
MessageDigest messageDigest = MessageDigest.getInstance("SHA1");
byte buf[] = new byte[8192];
int n;
InputStream inp = sap.getRangeStream();
while ((n = inp.read(buf)) > 0) {
messageDigest.update(buf, 0, n);
}
byte hash[] = messageDigest.digest();
PdfSigGenericPKCS sg = sap.getSigStandard();
PdfLiteral slit = (PdfLiteral)sg.get(PdfName.CONTENTS);
byte[] outc = new byte[(slit.getPosLength() - 2) / 2];
PdfPKCS7 sig = sg.getSigner();
sig.setExternalDigest(null, hash, null);
PdfDictionary dic = new PdfDictionary();
byte[] ssig = sig.getEncodedPKCS7();
System.arraycopy(ssig, 0, outc, 0, ssig.length);
dic.put(PdfName.CONTENTS, new PdfString(outc).setHexWriting(true));
sap.close(dic);

An example with an external hash and signature in Windows Certificate Mode

KeyStore ks = KeyStore.getInstance("pkcs12");
ks.load(new FileInputStream("my_private_key.pfx"), "my_password".toCharArray());
String alias = (String)ks.aliases().nextElement();
PrivateKey key = (PrivateKey)ks.getKey(alias, "my_password".toCharArray());
Certificate[] chain = ks.getCertificateChain(alias);
PdfReader reader = new PdfReader("original.pdf");
FileOutputStream fout = new FileOutputStream("signed.pdf");
PdfStamper stp = PdfStamper.createSignature(reader, fout, '\0');
PdfSignatureAppearance sap = stp.getSignatureAppearance();
sap.setCrypto(key, chain, null, PdfSignatureAppearance.WINCER_SIGNED);
sap.setReason("I'm the author");
sap.setLocation("Lisbon");
// comment next line to have an invisible signature
sap.setVisibleSignature(new Rectangle(100, 100, 200, 200), 1, null);
sap.setExternalDigest(new byte[128], new byte[20], "RSA");
sap.preClose();
MessageDigest messageDigest = MessageDigest.getInstance("SHA1");
byte buf[] = new byte[8192];
int n;
InputStream inp = sap.getRangeStream();
while ((n = inp.read(buf)) > 0) {
messageDigest.update(buf, 0, n);
}
byte hash[] = messageDigest.digest();
PdfSigGenericPKCS sg = sap.getSigStandard();
PdfLiteral slit = (PdfLiteral)sg.get(PdfName.CONTENTS);
byte[] outc = new byte[(slit.getPosLength() - 2) / 2];
PdfPKCS7 sig = sg.getSigner();
Signature sign = Signature.getInstance("SHA1withRSA");
sign.initSign(key);
sign.update(hash);
sig.setExternalDigest(sign.sign(), hash, "RSA");
PdfDictionary dic = new PdfDictionary();
byte[] ssig = sig.getEncodedPKCS7();
System.arraycopy(ssig, 0, outc, 0, ssig.length);
dic.put(PdfName.CONTENTS, new PdfString(outc).setHexWriting(true));
sap.close(dic);

An example with an external signature in Self Sign Mode

KeyStore ks = KeyStore.getInstance("pkcs12");
ks.load(new FileInputStream("my_private_key.pfx"), "my_password".toCharArray());
String alias = (String)ks.aliases().nextElement();
PrivateKey key = (PrivateKey)ks.getKey(alias, "my_password".toCharArray());
Certificate[] chain = ks.getCertificateChain(alias);
PdfReader reader = new PdfReader("original.pdf");
FileOutputStream fout = new FileOutputStream("signed.pdf");
PdfStamper stp = PdfStamper.createSignature(reader, fout, '\0');
PdfSignatureAppearance sap = stp.getSignatureAppearance();
sap.setCrypto(key, chain, null, PdfSignatureAppearance.SELF_SIGNED);
sap.setReason("I'm the author");
sap.setLocation("Lisbon");
// comment next line to have an invisible signature
sap.setVisibleSignature(new Rectangle(100, 100, 200, 200), 1, null);
sap.setExternalDigest(new byte[128], null, "RSA");
sap.preClose();
PdfPKCS7 sig = sap.getSigStandard().getSigner();
Signature sign = Signature.getInstance("SHA1withRSA");
sign.initSign(key);
byte buf[] = new byte[8192];
int n;
InputStream inp = sap.getRangeStream();
while ((n = inp.read(buf)) > 0) {
sign.update(buf, 0, n);
}
sig.setExternalDigest(sign.sign(), null, "RSA");
PdfDictionary dic = new PdfDictionary();
dic.put(PdfName.CONTENTS, new PdfString(sig.getEncodedPKCS1()).setHexWriting(true));
sap.close(dic);

How to sign with an external signature dictionary (DSE200 example)

An example with an external signature dictionary with the nCipher DSE 200 (this example is obsolete and should only be used with Acrobat 5)

PdfReader reader = new PdfReader("original.pdf");
FileOutputStream fout = new FileOutputStream("signed.pdf");
PdfStamper stp = PdfStamper.createSignature(reader, fout, '\0');
PdfSignatureAppearance sap = stp.getSignatureAppearance();
// comment next line to have an invisible signature
sap.setVisibleSignature(new Rectangle(100, 100, 200, 200), 1, null);
sap.setLayer2Text("This is some custom made text.");
PdfDictionary dic = new PdfDictionary();
dic.put(PdfName.FT, PdfName.SIG);
dic.put(PdfName.FILTER, new PdfName("nCipher.TimeSeal"));
dic.put(PdfName.SUBFILTER, new PdfName("ntse.dse.1"));
sap.setCryptoDictionary(dic);
HashMap exc = new HashMap();
exc.put(PdfName.CONTENTS, new Integer(0x1802));
exc.put(PdfName.M, new Integer(0x19));
exc.put(PdfName.NAME, new Integer(0x1f));
sap.preClose(exc);

MessageDigest messageDigest = MessageDigest.getInstance("MD5");
byte buf[] = new byte[8192];
int n;
InputStream inp = sap.getRangeStream();
while ((n = inp.read(buf)) > 0) {
messageDigest.update(buf, 0, n);
}
byte hash[] = messageDigest.digest();
// The DataImprint object is the "data to be time-stamped"
DataImprint dataImprint = new DataImprint();
dataImprint.setHashAlgorithm(new AlgorithmIdentifier(AlgorithmIdentifier.md5));
dataImprint.setHashedData(hash);

TimeStampRequest tsq = new TimeStampRequest();

tsq.setDataImprint(dataImprint);

// Set the nonce. It allows the client to verify the timeliness
// of the response when no trusted local clock is available.
Random rand = new Random(new Date().getTime());
tsq.setNonce(BigInteger.valueOf(rand.nextLong()));

// Request that the TSA signer certificate be included in the TimeStampToken.
// This allows a TimeStampToken signature to be verified without an
// external TSA certificate.
tsq.setCertReq(true);

// Set the request policy under which this TimeStampToken should be issued.
tsq.setReqPolicy(new PolicyIdentifier(PolicyIdentifier.serviceClass));

// Create the encoded request.
byte[] encodedTSQ = tsq.encodeRequest();

// Send the request to the Trusted Time StampServer
// and get the response (an encoded TimeStampToken)
TimeStampServerTCP tss = new TimeStampServerTCP("dse200 address");
byte[] encodedTST = tss.submitRequest(encodedTSQ, REQUEST_TIMEOUT);
// Decode the token to access the contained information
TimeStampToken tst = new TimeStampToken(encodedTST);

if(!tst.verifySignature())
throw new SignatureException("TimeStampToken signature invalid!");
TSTInfo tstInfo = tst.getTSTInfo();
GregorianCalendar cal = new GregorianCalendar(TimeZone.getTimeZone("GMT"));
cal.setTime(tstInfo.getDate());
StringTokenizer tk = new StringTokenizer(tstInfo.getTsaName(), ";");
String sn = "";
while (tk.hasMoreTokens()) {
String x;
if ((x = tk.nextToken()).startsWith("CN=")) {
sn = x.substring(3);
break;
}
}

PdfDictionary dic2 = new PdfDictionary();
dic2.put(PdfName.CONTENTS, new PdfString(encodedTST).setHexWriting(true));
dic2.put(PdfName.M, new PdfDate(cal));
dic2.put(PdfName.NAME, new PdfString(sn));
sap.close(dic2);

How to sign with an external signature dictionary using BouncyCastle CMS

An example with an external signature dictionary with the BouncyCastle CMS. The detached and encapsulated methods are presented.

import com.lowagie.text.Rectangle;
import com.lowagie.text.pdf.PdfDate;
import com.lowagie.text.pdf.PdfDictionary;
import com.lowagie.text.pdf.PdfName;
import com.lowagie.text.pdf.PdfPKCS7;
import com.lowagie.text.pdf.PdfReader;
import com.lowagie.text.pdf.PdfSignature;
import com.lowagie.text.pdf.PdfSignatureAppearance;
import com.lowagie.text.pdf.PdfStamper;
import com.lowagie.text.pdf.PdfString;
import java.io.ByteArrayOutputStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.GregorianCalendar;
import java.util.HashMap;
import org.bouncycastle.cms.CMSException;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import java.security.cert.Certificate;
import java.security.PrivateKey;
import java.security.KeyStore;
import java.security.MessageDigest;
import java.security.Security;
import java.security.cert.CertStore;
import java.security.cert.CollectionCertStoreParameters;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import org.bouncycastle.cms.CMSProcessable;
import org.bouncycastle.cms.CMSProcessableByteArray;
import org.bouncycastle.cms.CMSSignedData;
import org.bouncycastle.cms.CMSSignedDataGenerator;

public static void SignDetached() {
try {
Security.addProvider(new BouncyCastleProvider());
KeyStore ks = KeyStore.getInstance("PKCS12");
ks.load(new FileInputStream("my_private_key.pfx"), "password".toCharArray());
String alias = (String)ks.aliases().nextElement();
PrivateKey key = (PrivateKey)ks.getKey(alias, "password".toCharArray());
Certificate[] chain = ks.getCertificateChain(alias);

PdfReader reader = new PdfReader("original.pdf");
PdfStamper stp = PdfStamper.createSignature(reader, new FileOutputStream("signedDetached.pdf"), '\0');
PdfSignatureAppearance sap = stp.getSignatureAppearance();
sap.setVisibleSignature(new Rectangle(100, 100, 300, 200), 1, null);
sap.setSignDate(new GregorianCalendar());
sap.setCrypto(null, chain, null, null);
sap.setReason("I like to sign");
sap.setLocation("Universe");
sap.setAcro6Layers(true);
sap.setRender(PdfSignatureAppearance.SignatureRenderNameAndDescription);
PdfSignature dic = new PdfSignature(PdfName.ADOBE_PPKLITE, PdfName.ADBE_PKCS7_DETACHED);
dic.setDate(new PdfDate(sap.getSignDate()));
dic.setName(PdfPKCS7.getSubjectFields((X509Certificate)chain[0]).getField("CN"));
if (sap.getReason() != null)
dic.setReason(sap.getReason());
if (sap.getLocation() != null)
dic.setLocation(sap.getLocation());
sap.setCryptoDictionary(dic);
int csize = 4000;
HashMap exc = new HashMap();
exc.put(PdfName.CONTENTS, new Integer(csize * 2 + 2));
sap.preClose(exc);

CMSSignedDataGenerator generator = new CMSSignedDataGenerator();
generator.addSigner(key, (X509Certificate)chain[0], CMSSignedDataGenerator.DIGEST_SHA1);

ArrayList list = new ArrayList();
for (int i = 0; i < chain.length; i++) {
list.add(chain[i]);
}
CertStore chainStore = CertStore.getInstance("Collection", new CollectionCertStoreParameters(list), "BC");
generator.addCertificatesAndCRLs(chainStore);
CMSProcessable content = new CMSProcessableRange(sap);
CMSSignedData signedData = generator.generate(content, false, "BC");
byte[] pk = signedData.getEncoded();

byte[] outc = new byte[csize];

PdfDictionary dic2 = new PdfDictionary();

System.arraycopy(pk, 0, outc, 0, pk.length);

dic2.put(PdfName.CONTENTS, new PdfString(outc).setHexWriting(true));
sap.close(dic2);
}
catch (Exception e) {
e.printStackTrace();
}
}

public static void signEncapsulated() {
try {
Security.addProvider(new BouncyCastleProvider());
KeyStore ks = KeyStore.getInstance("PKCS12");
ks.load(new FileInputStream("my_private_key.pfx"), "password".toCharArray());
String alias = (String)ks.aliases().nextElement();
PrivateKey key = (PrivateKey)ks.getKey(alias, "password".toCharArray());
Certificate[] chain = ks.getCertificateChain(alias);

PdfReader reader = new PdfReader("original.pdf");
PdfStamper stp = PdfStamper.createSignature(reader, new FileOutputStream("signedEncapsulated.pdf"), '\0');
PdfSignatureAppearance sap = stp.getSignatureAppearance();
sap.setVisibleSignature(new Rectangle(100, 100, 300, 200), 1, null);
sap.setSignDate(new GregorianCalendar());
sap.setCrypto(null, chain, null, null);
sap.setReason("I like to sign");
sap.setLocation("Universe");
sap.setAcro6Layers(true);
sap.setRender(PdfSignatureAppearance.SignatureRenderNameAndDescription);
PdfSignature dic = new PdfSignature(PdfName.ADOBE_PPKMS, PdfName.ADBE_PKCS7_SHA1);
dic.setDate(new PdfDate(sap.getSignDate()));
dic.setName(PdfPKCS7.getSubjectFields((X509Certificate)chain[0]).getField("CN"));
if (sap.getReason() != null)
dic.setReason(sap.getReason());
if (sap.getLocation() != null)
dic.setLocation(sap.getLocation());
sap.setCryptoDictionary(dic);
int csize = 4000;
HashMap exc = new HashMap();
exc.put(PdfName.CONTENTS, new Integer(csize * 2 + 2));
sap.preClose(exc);

MessageDigest md = MessageDigest.getInstance("SHA1");
InputStream s = sap.getRangeStream();
int read = 0;
byte[] buff = new byte[8192];
while ((read = s.read(buff, 0, 8192)) > 0) {
md.update(buff, 0, read);
}

CMSSignedDataGenerator generator = new CMSSignedDataGenerator();
generator.addSigner(key, (X509Certificate)chain[0], CMSSignedDataGenerator.DIGEST_SHA1);

ArrayList list = new ArrayList();
for (int i = 0; i < chain.length; i++) {
list.add(chain[i]);
}
CertStore chainStore = CertStore.getInstance("Collection", new
CollectionCertStoreParameters(list), "BC");
generator.addCertificatesAndCRLs(chainStore);
CMSProcessable content = new CMSProcessableByteArray(md.digest());
CMSSignedData signedData = generator.generate(content, true, "BC");
byte[] pk = signedData.getEncoded();

byte[] outc = new byte[csize];

PdfDictionary dic2 = new PdfDictionary();

System.arraycopy(pk, 0, outc, 0, pk.length);

dic2.put(PdfName.CONTENTS, new PdfString(outc).setHexWriting(true));
sap.close(dic2);
}
catch (Exception e) {
e.printStackTrace();
}
}

public class CMSProcessableRange implements CMSProcessable {
private PdfSignatureAppearance sap;
private byte[] buf = new byte[8192];

public CMSProcessableRange(PdfSignatureAppearance sap) {
this.sap = sap;
}

public void write(OutputStream outputStream) throws IOException, CMSException {
InputStream s = sap.getRangeStream();
ByteArrayOutputStream ss = new ByteArrayOutputStream();
int read = 0;
while ((read = s.read(buf, 0, buf.length)) > 0) {
outputStream.write(buf, 0, read);
}
}

public Object getContent() {
return sap;
}
}

How to sign with an external signature dictionary (authenticatedAttributes example)

An example with an external signature dictionary with authenticatedAttributes

KeyStore ks = KeyStore.getInstance("pkcs12");
ks.load(new FileInputStream("my_private_key.pfx"),
"my_password".toCharArray());
String alias = (String)ks.aliases().nextElement();
PrivateKey key = (PrivateKey)ks.getKey(alias, "password".toCharArray());
Certificate[] chain = ks.getCertificateChain(alias);

PdfReader reader = new PdfReader("original.pdf");
FileOutputStream fout = new FileOutputStream("signed.pdf");
PdfStamper stp = PdfStamper.createSignature(reader, fout, '\0');
PdfSignatureAppearance sap = stp.getSignatureAppearance();
// comment next line to have an invisible signature
sap.setVisibleSignature(new Rectangle(100, 100, 200, 200), 1, null);
sap.setLayer2Text("This is some custom made text.\n\nDate: some date");
Calendar cal = Calendar.getInstance();
PdfDictionary dic = new PdfDictionary();
dic.put(PdfName.FT, PdfName.SIG);
dic.put(PdfName.FILTER, new PdfName("SAFE.PPKSF"));
dic.put(PdfName.SUBFILTER, new PdfName("adbe.pkcs7.detached"));
dic.put(PdfName.M, new PdfDate(cal));
dic.put(PdfName.NAME, new
PdfString(PdfPKCS7.getSubjectFields((X509Certificate)chain[0]).getField("CN")));
sap.setCryptoDictionary(dic);
HashMap exc = new HashMap();
exc.put(PdfName.CONTENTS, new Integer(0x2502));
sap.preClose(exc);

PdfPKCS7 pk7 = new PdfPKCS7(key, chain, null, "SHA1", null, false);
MessageDigest messageDigest = MessageDigest.getInstance("SHA1");
byte buf[] = new byte[8192];
int n;
InputStream inp = sap.getRangeStream();
while ((n = inp.read(buf)) > 0) {
messageDigest.update(buf, 0, n);
}
byte hash[] = messageDigest.digest();
byte sh[] = pk7.getAuthenticatedAttributeBytes(hash, cal);
pk7.update(sh, 0, sh.length);
PdfDictionary dic2 = new PdfDictionary();
byte sg[] = pk7.getEncodedPKCS7(hash, cal);
byte out[] = new byte[0x2500 / 2];
System.arraycopy(sg, 0, out, 0, sg.length);
dic2.put(PdfName.CONTENTS, new PdfString(out).setHexWriting(true));
sap.close(dic2);

How to verify with iTextSharp

How to verify a signature using the certificates in the Windows certificate store using WSE.

using wse = Microsoft.Web.Services.Security.X509;
using Org.BouncyCastle.X509;

wse.X509CertificateStore store = new wse.X509CertificateStore(wse.X509CertificateStore.StoreProvider.System, wse.X509CertificateStore.StoreLocation.LocalMachine, wse.X509CertificateStore.RootStore);
store.Open();
wse.X509CertificateCollection c = store.Certificates;
System.Console.Out.WriteLine("There are " + c.Count + " certificates in the Windows certificate store");
X509CertificateParser parser = new X509CertificateParser();
ArrayList kall = new ArrayList();
foreach (wse.X509Certificate cert in c) {
X509Certificate c2 = parser.ReadCertificate(cert.GetRawCertData());
kall.Add(c2);
}
PdfReader reader = new PdfReader("signed.pdf");
AcroFields af = reader.AcroFields;
ArrayList names = af.GetSignatureNames();
for (int k = 0; k < names.Count; ++k) {
String name = (String)names[k];
System.Console.Out.WriteLine("Signature name: " + name);
System.Console.Out.WriteLine("Signature covers whole document: " + af.SignatureCoversWholeDocument(name));
System.Console.Out.WriteLine("Document revision: " + af.GetRevision(name) + " of " + af.TotalRevisions);
PdfPKCS7 pk = af.VerifySignature(name);
DateTime cal = pk.SignDate;
X509Certificate[] pkc = pk.Certificates;
System.Console.Out.WriteLine("Subject: " + PdfPKCS7.GetSubjectFields(pk.SigningCertificate).GetField("CN"));
System.Console.Out.WriteLine("Document modified: " + !pk.Verify());
Object[] fails = PdfPKCS7.VerifyCertificates(pkc, kall, null, cal);
if (fails == null)
System.Console.Out.WriteLine("Certificates verified against the KeyStore");
else
System.Console.Out.WriteLine("Certificate failed: " + fails[1]);
}

How to sign with a smartcard using an external signature dictionary with iTextSharp, CAPICOM and .NET 1.1

This example allows the signing with smartcards. The signing is done using the Windows mode and includes the hash in the signature. This example requires .NET 1.1 and CAPICOM 2.0 as a COM Interop (select the CAPICOM as a reference and Visual Studio will create the C# interfaces and classes).

using System;
using System.IO;
using System.Collections;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
using CAPICOM;
using iTextSharp.text;
using iTextSharp.text.pdf;

namespace CCapi {
class Class1 {
static void Main(string[] args) {
Certificate cer = GetCertificate();
Org.BouncyCastle.X509.X509Certificate[] chain = GetChain(cer);
PdfReader reader = new PdfReader("c:\\hello.pdf");
PdfStamper stp = PdfStamper.CreateSignature(reader, new FileStream("c:\\hello_hashedcapi.pdf", FileMode.Create), '\0');
PdfSignatureAppearance sap = stp.SignatureAppearance;
sap.SetVisibleSignature(new Rectangle(100, 100, 300, 200), 1, null);
sap.SignDate = DateTime.Now;
sap.SetCrypto(null, chain, null, null);
sap.Reason = "I like to sign";
sap.Location = "Universe";
sap.Acro6Layers = true;
sap.Render = PdfSignatureAppearance.SignatureRender.NameAndDescription;
PdfSignature dic = new PdfSignature(PdfName.ADOBE_PPKMS, PdfName.ADBE_PKCS7_SHA1);
dic.Date = new PdfDate(sap.SignDate);
dic.Name = PdfPKCS7.GetSubjectFields(chain[0]).GetField("CN");
if (sap.Reason != null)
dic.Reason = sap.Reason;
if (sap.Location != null)
dic.Location = sap.Location;
sap.CryptoDictionary = dic;
int csize = 4000;
Hashtable exc = new Hashtable();
exc[PdfName.CONTENTS] = csize * 2 + 2;
sap.PreClose(exc);

HashAlgorithm sha = new SHA1CryptoServiceProvider();

Stream s = sap.RangeStream;
int read = 0;
byte[] buff = new byte[8192];
while ((read = s.Read(buff, 0, 8192)) > 0) {
sha.TransformBlock(buff, 0, read, buff, 0);
}
sha.TransformFinalBlock(buff, 0, 0);
byte[] pk = SigMsg(sha.Hash, cer, false);
byte[] outc = new byte[csize];

PdfDictionary dic2 = new PdfDictionary();

Array.Copy(pk, 0, outc, 0, pk.Length);

dic2.Put(PdfName.CONTENTS, new PdfString(outc).SetHexWriting(true));
sap.Close(dic2);
}

public static byte[] SigMsg(byte[] msg, Certificate cert, bool detached) {
SignerClass signer = new SignerClass();

// check for private key in the certificate
if (cert.HasPrivateKey())
signer.Certificate = cert;
else
throw new ArgumentException("Certificate has no private key");
UtilitiesClass ut = new UtilitiesClass();
SignedDataClass sd = new SignedDataClass();
string gg = ut.ByteArrayToBinaryString(msg);
sd.Content = gg;
string signedData = sd.Sign(signer, detached, CAPICOM_ENCODING_TYPE.CAPICOM_ENCODE_BASE64);
return Convert.FromBase64String(signedData);
}

public static Certificate GetCertificate() {
StoreClass store = new StoreClass();
// can also be CAPICOM_STORE_LOCATION.CAPICOM_CURRENT_USER_STORE
store.Open(CAPICOM_STORE_LOCATION.CAPICOM_SMART_CARD_USER_STORE,
"My", CAPICOM_STORE_OPEN_MODE.CAPICOM_STORE_OPEN_READ_ONLY);
Certificate cc = null;
// choose the certificate. May also choose by serial id, etc.
foreach (Certificate cert in store.Certificates) {
if (cert.SubjectName.IndexOf("Paulo Soares") > 0) {
cc = cert;
break;
}
}
return cc;
}

public static Org.BouncyCastle.X509.X509Certificate[] GetChain(Certificate cer) {
ICertContext context = cer as ICertContext;
IntPtr ptr = new IntPtr(context.CertContext);
X509Certificate xCert = new X509Certificate(ptr);
Org.BouncyCastle.X509.X509CertificateParser cp =
new Org.BouncyCastle.X509.X509CertificateParser();
Org.BouncyCastle.X509.X509Certificate[] chain =
new Org.BouncyCastle.X509.X509Certificate[]{cp.ReadCertificate(xCert.GetRawCertData())};
return chain;
}
}
}

How to sign with a smartcard using an external signature dictionary with iTextSharp and .NET 2.0

This example allows the signing with smartcards. Two functions are provided: SignHashed() signs using the Windows mode and includes the hash in the signature and SignDetached() signs in detached mode but needs more memory to do it. This example requires .NET 2.0.

using System;
using System.Collections;
using System.IO;
using System.Security.Cryptography.X509Certificates;
using System.Security.Cryptography.Pkcs;
using System.Security.Cryptography;
using iTextSharp.text;
using iTextSharp.text.pdf;

public static void SignHashed() {
X509Certificate2 card = GetCertificate();
Org.BouncyCastle.X509.X509CertificateParser cp = new Org.BouncyCastle.X509.X509CertificateParser();
Org.BouncyCastle.X509.X509Certificate[] chain = new Org.BouncyCastle.X509.X509Certificate[]{cp.ReadCertificate(card.RawData)};

PdfReader reader = new PdfReader("c:\\hello.pdf");
PdfStamper stp = PdfStamper.CreateSignature(reader, new FileStream("c:\\hello_hashed.pdf", FileMode.Create), '\0');
PdfSignatureAppearance sap = stp.SignatureAppearance;
sap.SetVisibleSignature(new Rectangle(100, 100, 300, 200), 1, null);
sap.SignDate = DateTime.Now;
sap.SetCrypto(null, chain, null, null);
sap.Reason = "I like to sign";
sap.Location = "Universe";
sap.Acro6Layers = true;
sap.Render = PdfSignatureAppearance.SignatureRender.NameAndDescription;
PdfSignature dic = new PdfSignature(PdfName.ADOBE_PPKMS, PdfName.ADBE_PKCS7_SHA1);
dic.Date = new PdfDate(sap.SignDate);
dic.Name = PdfPKCS7.GetSubjectFields(chain[0]).GetField("CN");
if (sap.Reason != null)
dic.Reason = sap.Reason;
if (sap.Location != null)
dic.Location = sap.Location;
sap.CryptoDictionary = dic;
int csize = 4000;
Hashtable exc = new Hashtable();
exc[PdfName.CONTENTS] = csize * 2 + 2;
sap.PreClose(exc);

HashAlgorithm sha = new SHA1CryptoServiceProvider();

Stream s = sap.RangeStream;
int read = 0;
byte[] buff = new byte[8192];
while ((read = s.Read(buff, 0, 8192)) > 0) {
sha.TransformBlock(buff, 0, read, buff, 0);
}
sha.TransformFinalBlock(buff, 0, 0);
byte[] pk = SignMsg(sha.Hash, card, false);

byte[] outc = new byte[csize];

PdfDictionary dic2 = new PdfDictionary();

Array.Copy(pk, 0, outc, 0, pk.Length);

dic2.Put(PdfName.CONTENTS, new PdfString(outc).SetHexWriting(true));
sap.Close(dic2);
}

public static void SignDetached() {
X509Certificate2 card = GetCertificate();
Org.BouncyCastle.X509.X509CertificateParser cp = new Org.BouncyCastle.X509.X509CertificateParser();
Org.BouncyCastle.X509.X509Certificate[] chain = new Org.BouncyCastle.X509.X509Certificate[]{cp.ReadCertificate(card.RawData)};

PdfReader reader = new PdfReader("c:\\hello.pdf");
PdfStamper stp = PdfStamper.CreateSignature(reader, new FileStream("c:\\hello_detached.pdf", FileMode.Create), '\0');
PdfSignatureAppearance sap = stp.SignatureAppearance;
sap.SetVisibleSignature(new Rectangle(100, 100, 300, 200), 1, null);
sap.SignDate = DateTime.Now;
sap.SetCrypto(null, chain, null, null);
sap.Reason = "I like to sign";
sap.Location = "Universe";
sap.Acro6Layers = true;
sap.Render = PdfSignatureAppearance.SignatureRender.NameAndDescription;
PdfSignature dic = new PdfSignature(PdfName.ADOBE_PPKLITE, PdfName.ADBE_PKCS7_DETACHED);
dic.Date = new PdfDate(sap.SignDate);
dic.Name = PdfPKCS7.GetSubjectFields(chain[0]).GetField("CN");
if (sap.Reason != null)
dic.Reason = sap.Reason;
if (sap.Location != null)
dic.Location = sap.Location;
sap.CryptoDictionary = dic;
int csize = 4000;
Hashtable exc = new Hashtable();
exc[PdfName.CONTENTS] = csize * 2 + 2;
sap.PreClose(exc);

Stream s = sap.RangeStream;
MemoryStream ss = new MemoryStream();
int read = 0;
byte[] buff = new byte[8192];
while ((read = s.Read(buff, 0, 8192)) > 0) {
ss.Write(buff, 0, read);
}
byte[] pk = SignMsg(ss.ToArray(), card, true);

byte[] outc = new byte[csize];

PdfDictionary dic2 = new PdfDictionary();

Array.Copy(pk, 0, outc, 0, pk.Length);

dic2.Put(PdfName.CONTENTS, new PdfString(outc).SetHexWriting(true));
sap.Close(dic2);
}

// Sign the message with the private key of the signer.
static public byte[] SignMsg(Byte[] msg, X509Certificate2 signerCert, bool detached) {
// Place message in a ContentInfo object.
// This is required to build a SignedCms object.
ContentInfo contentInfo = new ContentInfo(msg);

// Instantiate SignedCms object with the ContentInfo above.
// Has default SubjectIdentifierType IssuerAndSerialNumber.
SignedCms signedCms = new SignedCms(contentInfo, detached);

// Formulate a CmsSigner object for the signer.
CmsSigner cmsSigner = new CmsSigner(signerCert);

// Include the following line if the top certificate in the
// smartcard is not in the trusted list.
cmsSigner.IncludeOption = X509IncludeOption.EndCertOnly;

// Sign the CMS/PKCS #7 message. The second argument is
// needed to ask for the pin.
signedCms.ComputeSignature(cmsSigner, false);

// Encode the CMS/PKCS #7 message.
return signedCms.Encode();
}

public static X509Certificate2 GetCertificate() {
X509Store st = new X509Store(StoreName.My, StoreLocation.CurrentUser);
st.Open(OpenFlags.ReadOnly);
X509Certificate2Collection col = st.Certificates;
X509Certificate2 card = null;
X509Certificate2Collection sel = X509Certificate2UI.SelectFromCollection(col, "Certificates", "Select one to sign", X509SelectionFlag.SingleSelection);
if (sel.Count > 0) {
X509Certificate2Enumerator en = sel.GetEnumerator();
en.MoveNext();
card = en.Current;
}
st.Close();
return card;
}

Friday, May 23, 2008

Restarting a remote computer

If you want to restart a computer which you have connected via Windows Remote Desktop do this.

Start >> Run >> shutdown -r -f

The machine would be restarted after 30 seconds automatically

There are lots of other options available with shutdown command. You can view all the available options by typing "shutdown /?" in the command window.

Saturday, April 19, 2008

The New Builder Pattern

The New Builder Pattern

The idea

I like to create immutable objects, especially after reading Josh Bloch's excellent "Effective Java" book. If an object is immutable, it has only one possible state and it is a stable one, so once you successfully build an object, you don't need to care about state transitions that can make your object unstable or corrupted. And immutable objects can be shared even in a multithreaded application. There are many other pros of immutability (you can read some of them here).

There is a classical way of making immutable objects in Java which consists of making all fields final (and private, of course), using only constructors to modify them (so that the only moment when a field is modified is during its construction) and making the class final (to avoid adding "setter" methods to subclasses). When you only have a couple of fields, that's fine, but when you have many of them you end up with a constructor with many arguments, which is ugly and difficult to use. If you have optional parameters, you can have a constructor with all the parameters and some other shorter constructors that have the mandatory parameters and some optional ones, that invoke the big constructor, like this:


public class Foo {

private final String mandatoryOne;
private final String mandatoryTwo;
private final String optionalOne;
private final String optionalTwo;

public Foo(String mOne, String mTwo, String optOne, String optTwo){
this.mandatoryOne = mOne;
this.mandatoryTwo = mTwo;
this.optionalOne = optOne;
this.optionalTwo = optTwo;
}

public Foo(String mOne, String mTwo, String optOne){
this(mOne, mTwo, optOne, null);
}
...
}


This can be a bit messy when you add more optional parameters, you end up with a lot of constructors like these and it has a lot of boilerplate code.The use of setters for the optional parameters is not an option, because this leads to non immutable objects (some object can change the state of your object with one of those setter methods).
Some time ago, thinking about this problem, I thought a solution could be to use a Javabean object, with one setter per field (even for the mandatory ones), but with a kind of "seal" method, that would "mark" the object as built and since that moment, an IllegalStateException would be thrown if a setter was called. Nevertheless, I wasn't very satisfied with this approach, because the setter methods that sometimes can be called and sometimes not would be confusing for the caller.

Then I found the New Builder pattern, explained by Josh Bloch in this PDF presentation, which is different from the original GoF Builder pattern. This pattern uses a public inner static class as a builder. The constructor of the original class is made private, so the only way to build objects is with the inner builder class. The builder has a setter method for each optional parameter and uses a fluent idiom that allows chaining of these method calls. I like this pattern a lot, because it solves the problem elegantly and effectively.

The implementation

In Josh Bloch's presentation there wasn't a detailed implementation of the pattern, although it was very clear the idea and the intention so I have searched for it in the Internet.

In Richard Hansen's blog you can find an implementation that seems to be more close to what Josh explains: the builder is a static nested class of the class from which it has to make instances, the builder's constructor is public (so you invoke the builder with 'new'), and the builder has the same fields as its enclosing class. The 'build()' method copies the content of the builder's fields into a new instance of the enclosing class. What I don't like about this implementation is this duplication of fields (for each field in the original class you have a duplicate field in the builder).

In Robbie Vanbrabant's blog there is a variation of this pattern, which avoids the boilerplate code using a base class for the builder and some reflection to build the object from the builder. I don't like the use of an interface for the builder, because that way you can't add a new optional parameter without breaking existing code that uses the builder (if you change the signature of a public interface the classes that use it have to change their code to implement the new methods). Also, I don't like the use of reflection because it's slower than the normal access to fields, but I do like the way it avoids duplication of fields in the builder.

The implementation I like most is the one found in Mario Hochreiter's blog. The builder is a nested public static class, but it changes the fields of its enclosing class directly, it doesn't use duplicates. It doesn't use reflection and the builder is a class, not an interface. The only problem I see is that, in theory, with a reference to a builder, you can change the state of the object it built, so you don't have the guarantee that the object is immutable. So I would add a check before each "setter" of the builder that would throw an IllegalStateException if the object has been already built and a check before the 'build' method itself to ensure the object is not built more than once. Also, I would make the mandatory parameters final.

So, with the example of Mario, I would implement this pattern this way:


public class ID3Tag {

private final String title;
private final String artist;
private String album;
private int albumTrack;
private String comment;

public static class Builder {

private boolean isBuilt = false;
private ID3Tag id3tag;

public Builder(String title, String artist) {
id3tag = new ID3Tag(title, artist);
}

public Builder album(String val) {
if (isBuilt){
throw new IllegalStateException("The object cannot be modified after built");
}
id3tag.album = val;
return this;
}

public Builder albumTrack(int val) {
if (isBuilt){
throw new IllegalStateException("The object cannot be modified after built");
}
id3tag.albumTrack = val;
return this;
}

public Builder comment(String val) {
if (isBuilt){
throw new IllegalStateException("The object cannot be modified after built");
}
id3tag.comment = val;
return this;
}
// ... a lot more optional parameters

public ID3Tag build() {
if (isBuilt){
throw new IllegalStateException("The object cannot be built twice");
}
isBuilt = true;
return id3tag;
}
}

private ID3Tag(String title, String artist) {
this.title = title;
this.artist = artist;
}
}


The usage of this class would be:

ID3Tag tag = new ID3Tag.Builder("My Title", "My author")
.comment("Great song").build();



I have found a similar pattern, called the Essence pattern, described here and here by Dr Herbie. This pattern uses direct access to the fields of the builder (like in a C++ structure) instead of using "setter" methods and it doesn't use "chaining" of modifications like in the New Builder Pattern ("...builder.option1(value1).option2(value2)...").

Initialization on demand holder idiom

Initialization on demand holder idiom

In software engineering, the Initialization on Demand Holder idiom (design pattern) is a lazy-loaded singleton. The idiom can be implemented in both single-threaded/serial and concurrent environments, but care must be taken to correctly implement the idiom under concurrent conditions.

Example Java Implementation

This implementation from Bill Pugh (see links below) is a well-performing and concurrent implementation valid in all versions of Java. The original implementation from Bill Pugh has been modified to reduce the scope of LazyHolder.something to package and to make the field final.

public class Something {
private Something() {
}

private static class LazyHolder {
private static final Something something = new Something();
}

public static Something getInstance() {
return LazyHolder.something;
}
}

How it works

The implementation relies on the well-specified initialization phase of execution within the Java Virtual Machine (JVM); see section 12.4 of Java Language Specification (JLS) for details.

When the class Something is loaded by the JVM, the class goes through initialization. Since the class does not have any static variables to initialize, the initialization completes trivially. The static class definition LazyHolder within it is not initialized until the JVM determines that LazyHolder must be executed. The static class LazyHolder is only executed when the static method getInstance is invoked on the class Something, and the first time this happens the JVM will load and initialize the LazyHolder class. The initialization of the LazyHolder class results in static variable something being initialized by executing the (private) constructor for the outer class Something. Since the class initialization phase is guaranteed by the JLS to be serial, i.e., non-concurrent, no further synchronization is required in the static getInstance method during loading and initialization. And since the initialization phase writes the static variable something in a serial operation, all subsequent concurrent invocations of the getInstance will return the same correctly initialized something without incurring any additional synchronization overhead.

When to use it

Use this pattern if the initialization of the class is expensive and it cannot be done safely at class-loading time and the initialization is highly concurrent. The crux of the pattern is the safe removal of the synchronization overhead associated with accessing a singleton instance.

A Java Builder Pattern

A Java Builder Pattern

There's a Builder pattern that Joshua Bloch has briefly described in a couple of his "Effective Java Reloaded" sessions at Java One. This Builder is not necessarily a replacement for the original design pattern. The problems this Builder pattern can solve are too many constructors, too many constructor parameters, and over use of setters to create an object.


Here are some examples of the pattern in use. These examples create various Widgets with two required properties and several optional ones -




Widget x = new Widget.Builder("1", 1.0).
model("1").build();
Widget y = new Widget.Builder("2", 2.0).
model("2").manufacturer("222").
serialNumber("12345").build();
Widget z = new Widget.Builder("3", 4.0).
manufacturer("333").
serialNumber("54321").build();


The basic idea behind the pattern is to limit the number of constructor parameters and avoid the use of setter methods. Constructors with too many parameters, especially optional ones, are ugly and hard to use. Multiple constructors for different modes are confusing. Setter methods add clutter and force an object to be mutable. Here is an class skeleton of the pattern -



public class Widget {
public static class Builder {
public Builder(String name, double price) { ... }
public Widget build() { ... }
public Builder manufacturer(String value) { ... }
public Builder serialNumber(String value) { ... }
public Builder model(String value) { ... }
}

private Widget(Builder builder) { ... }
}


Notice that Widget has no public constructor and no setters and that the only way to create a Widget is using the static inner class Widget.Builder. Widget.Builder has a constructor that takes the required properties of Widget. Widget's optional properties can be set using optional property methods on the Widget.Builder. The property methods of Widget.Builder return a reference to the builder so method calls can be chained.


A really nice feature of this pattern is the ability to do pre-creation validation of an object state. When setters are used to set object state during creation it is virtually impossible to guarantee that object has been properly created.


Here is the full source for Widget and its Builder -



public class Widget {
public static class Builder {
private String name;
private String model;
private String serialNumber;
private double price;
private String manufacturer;

public Builder(String name, double price) {
this.name = name;
this.price = price;
}

public Widget build() {
// any pre-creation validation here
Widget result = new Widget(name, price);
result.model = model;
result.serialNumber = serialNumber;
result.manufacturer = manufacturer;
return result;
}

public Builder manufacturer(String value) {
this.manufacturer = value;
return this;
}

public Builder serialNumber(String value) {
this.serialNumber = value;
return this;
}

public Builder model(String value) {
this.model = value;
return this;
}
}

private String name;
private String model;
private String serialNumber;
private double price;
private String manufacturer;

/**
* Creates an immutable widget instance.
*/
private Widget(String name, double price) {
this.name = name;
this.price = price;
}

public String toString() {
return super.toString() + " {"
+ "name="
+ getName()
+ " model="
+ getModel()
+ " serialNumber="
+ getSerialNumber()
+ " price="
+ getPrice()
+ " manufacturer="
+ getManufacturer()
+ "}";
}

public String getManufacturer() {
return manufacturer;
}

public String getModel() {
return model;
}

public String getName() {
return name;
}

public double getPrice() {
return price;
}

public String getSerialNumber() {
return serialNumber;
}
}


Notice that Widget's private constructor takes the required properties and that the Builder sets the optional properties. Another thing to note is that widget is an immutable object as implemented.