%begin{latexonly}
\newif\ifpdf
\ifx\pdfoutput\undefined
\pdffalse
\else
\pdfoutput=1
\pdftrue
\fi

% Change this as needed :
%   - a4paper to your paper format
%   - the document class to your need (book, article, ...)
\ifpdf
\documentclass[a4paper, 12pt, pdftex]{report}
\else
%end{latexonly}
\documentclass[a4paper, 12pt, dvips]{report}
%begin{latexonly}
\fi
%end{latexonly}

% The packages we need
\usepackage{verbatim}
\usepackage{moreverb}
\usepackage{url}
\usepackage{tabularx}
\usepackage[final]{graphicx}
\usepackage[hyperindex,breaklinks=true,pdfborder={0 0 0}]{hyperref}
%begin{latexonly}
\ifpdf
\hypersetup{colorlinks=true,linkcolor=blue,urlcolor=blue,citecolor=red}
\fi
%end{latexonly}
\usepackage{html}
\begin{htmlonly}
\newcommand{\href}[2]{\htmladdnormallink{#2}{#1}}
\end{htmlonly}

% block style paragraphs tend to look better in technical docs
\parindent=0in
\parskip=10pt

\begin{document}

% Split Jar Specification

\appendix

\chapter{Split Jars and MANIFEST Extensions}

The jar file specification allows archiving and packaging classes and
resources, but is limited in size. To overcome these limitations with
minimal changes to the jar format we create a set of jar files and add
new key-value attributes to the jar MANIFEST. These attributes
indicate now many jars are in the set, and which, if any, files are
split across multiple jars, and which jars they are contained in.

\section{Motivations and Limitations}

Java's zip implementation limited to \~{}2GB. The problem is not solved
by zip64 extensions (which will allow larger files), when medium limitations
restrict the jar size. There must be a way to split the archive into
multiple files, and indeed, the individual entries must be split
across jars.

A ``Split Jar'' is a set of normal jar files, one being the
\emph{primary} jar, and zero or more \emph{secondary} jars. The
Primary jar file has additional manifest attributes to help
reconstruct the data. Entries may or may not be split across multiple
jars, and need to be spliced back together upon extraction. Secondary
jar names derive from the basename of the primary jar, and each
segment of a split entry, shares a basename derived from the original
entry name.

Segments of a split entry need not be in separate jars. Thus if a jar
is split to deal with media limitations, all the resulting jars may be
combined into a single primary jar, as long as the Manifest is
correctly updated.

A major benefit to this format is that the split archive contents can
be recovered manually by extracting the contents of all jar files in
the set, and simply concatenating the segments of the split entries.

Entry names in the primary and secondary jars must not conflict, so
that together they represent a single archive. This includes the
generated names of split entry segments. This ensures that each jar in
a split archive may be extracted to the same location without risk of
loosing data. Split file segments are then be concatenated manually or
by automation to get the original data set. The manifest should always
be consulted to ensure that files which look like split entry segments
should actually be spliced together. It is possible that the files
were intended to be part of the archive (See ``Naming Conventions'',
for name conflict resolution).

All segments of a split jar are given generated names so that normal
jar tools will never unpack the original file. This ensures that no
unsuspecting user mistakenly uses a truncated, partial file.

\subsection{Warnings}

Signing jar entries which have been split has not been addressed.

Files can not be compressed directly into streams when there are
potential name conflicts with the generated segment names. This
requires that robust tools collect a list of files to be added, and
determine any conflicts first to avoid the issue (See Naming
Conventions: Entry Names).

Adding files to existing split jars may also have problems with name
conflicts.

\section{Naming Conventions}

A primary goal for this design is to allow split jars to be created
and unpacked manually with minimal problems. This is accomplished by
using a naming convention which lends to visual reconstruction. When a
jar file must be split into multiple segments, there is a primary
file, and multiple secondary jars with a common name. When an entry
within the set of jars must be split, \emph{each} segment is given
a numbered suffix.

\subsection{Jar File Names}

For the primary jar \texttt{\emph{basename}.jar}, the names of
secondary jars must always be \texttt{\emph{basename}.split\#.jar}
where \texttt{\#} is an integer \emph{secondary jar ID} starting at
\texttt{1}. Left padded zeros in the ID are ignored, and encouraged
to allow lexicographical sorting. The jars can be renamed, as long as
the \emph{basename} is the same for all, and the suffixes
(\texttt{.split\#.jar}) remain the same. All entries within the set
must be unique.

\subsection{Jar Entry Names}

For the split entry named \texttt{\emph{basename}} (including
suffixes), all segments are named using the template:
\texttt{\emph{basename}}\texttt{.---\#.\~{}}, where \texttt{\#} is an integer
\emph{segment ID} starting at \texttt{0}. These segments are
rejoined by concatenating the segments in numeric order, to a file
named \texttt{basename}. The template is recorded in the \emph{main} section
of the manifest.

In the rare case where an entry is split, and the name of a real entry
may conflicts with a generated segment name, a non-default suffix
template is used. In Our case, all of the generated segments will have
'\texttt{\~{}}' characters appended, as needed, to eliminate potential
conflicts. This non-default template is recorded in the
\emph{per-entry} section of the manifest for the split entry.

Non-default suffixes are used for all \emph{potential} conflicts even in
cases where there is no actual conflict.

\begin{itemize}
  \item When the split entry does not generate enough segments to
        conflict, but the suffix matches the default template.
  \item When the conflicting real entry must also be split, thus its
        actual entries use generated suffixes.
\end{itemize}\

Examples are given below.

Other tools implementing split jars may (though are not encouraged to)
use different suffixes, though they must have numeric segment replaced
by '\#' in the manifest. Tools must sort these numerically, not
lexicographically as ``2'' is generally greater than ``10''
lexicographically. However, tools are encouraged to zero padding names,
as needed, so that lexicographic sorting is correct.

\section{Manifest Attributes}

To minimize changes needed to implement the split jar, we simply add
attributes to the manifest. Additional attributes are ignored by other
jar tools, so the only consequences is that files split files, and
files completely located in secondary jars will not be available to
them.

To prevent adding too much space overhead, and allow jar files to be
renamed, the entries are kept minimalistic.

\subsection{Main Section Attributes}

Two attribute are added to indicate the number of secondary jars, and
the default suffix added to the segments of split files.

% TODO: make this like an html <dl><dd>... <dt> ...</dl>
\begin{itemize}
  \item \texttt{Split-Jar-Secondary-Count}: The number of secondary jars
        in the set.
  \item \texttt{Split-Jar-Secondary-Suffix}: the suffix template
        inserted prior to the \texttt{.jar} suffix typical of jar
        files, to make the names of secondary jar file in the set;
        typically \texttt{.split\#}.
  \item \texttt{Split-Entry-Suffix}: the suffix template appended to
        an entry name, to name each of the entries constituent parts;
        typically \texttt{.---\#.\~{}}. The \# char indicates the
        location of the numeric value.  This cannot currently be
        changed.
\end{itemize}

\subsection{Per-Entry Section Attributes}

Only files which are split require an attributes in the manifest. A
space separated list of integers is recorded; one for each jar
containing a segment of the entry. Entries which have a segment in the
primary jar file, indicate this with the id \texttt{0}.

No restriction is placed on the order of the entries, or the IDs of the
jar in which any segment is contained.

% TODO: make this like an html <dl><dd>... <dt> ...</dl>
\begin{itemize}
  \item \texttt{Split-Entry-Jar-IDs}: A space separated set of
        secondary jar IDs which contains the segments of the
        entry. Essentially a list of integers.
  \item \texttt{Split-Entry-Suffix}: Overrides the default
        Split-Entry-Suffix specified in the Main-Attributes. Needed
        when one (or more) '\texttt{\~{}}' chars are appended due to name
        conflict with real entries.  This is not strictly necessary,
        as simply knowing the basename and unpacking all jars would
        allow the suffix to be determined, but is included to conserve
        processing. This is currently not user configurable.
\end{itemize}

\section{Examples}

Two examples, one simple, and another cluttered with pathological
cases. Notice that the jar ID number and segment of a split entry have
no correlation. In most applications, there will seldom be more than
two segments in a single file: the end of the last entry to the
previous jar, and maybe the last entry of this jar, which is continued
in the next. The examples aren't so well organized though. :-)

\subsection{Basic Example}

% TODO: format for tex
TODO: format for TeX

Jar to Create
-------------
    example.jar

Files to Compress
-----------------
    movie.mpeg
    README
    song.mp3
    text.txt

Entries in Jars
---------------
    example.jar           movie.mpeg.---0.\~{}
                          README
    
    example.split1.jar    movie.mpeg.---1.\~{}
                          song.mp3.---0.\~{}
    
    example.split2.jar    movie.mpeg.---2.\~{}
                          song.mp3.---1.\~{}
                          text.txt

MANIFEST (primary jar only)
---------------------------
    Manifest-Version: 1.0
    Created-By: 1.4.2\_04-b05 (Sun Microsystems Inc.)
    Built-By: IzPack 1.6.0
    Main-Class: com.izforge.izpack.installer.Installer
    Split-Jar-Secondary-Count: 2
    Split-Entry-Suffix: .---\#.\~{}
    
    movie.mpg
    Split-Entry-Jar-IDs: 0 1 2
    
    song.mp3
    Split-Entry-Jar-IDs: 1 2

\subsection{Name Conflicts}

Pathological example showing name conflict resolution.  Includes

\begin{itemize}
  \item Direct conflict with real archive file
        (\texttt{foo...}).

  \item Indirect conflict with file by suffix template only
        (\texttt{bar...}).

  \item Conflict with real archive file that is also split. Due to
        both being split, there would be no name conflict amongst jar
        entries, however The default suffix is not used anyway
        (\texttt{yin...}).

  \item A \emph{near} conflict, just to be annoying. Normal behavior
        (\texttt{chi...}).

  \item Files which look like segments of a split file, but are not,
        requiring manifest to know the difference (\texttt{zig...}).
\end{itemize}

\begin{verbatim}

Jar to Create
-------------
    example.jar

Files to Compress
-----------------
    foo.dat
    foo.dat.---0.~{} .... Extremely unlikely that these would exist,
                        much less need to be archived. Provided as an
                        example.
    bar.dat
    bar.dat.---555.~{} .. Another unlikely case which would not conflict
                        (assume bar.dat is split into only 2 segments)
                        except for the suffix template.
    yin.dat
    yin.dat.---2.~{} .... Yet another template only conflict
                        conflicting file needs to be split.
    chi.dat
    chi.dat.---0.~{}~{} ... No potential conflict.
    zig.dat.---0.~{} .... Files to be archived as they are, but not
    zig.dat.---1.~{}      intended to be spliced back together.

Entries in Jars
---------------
    example.jar           foo.dat.---0.~{}
                          foo.dat.---0.~{}~{}
                          bar.dat.---555.~{}
                          bar.dat.---0.~{}~{}
                          yin.dat.---0.~{}~{}
                          yin.dat.---2.~{}.---0.~{}
                          chi.dat.---0.~{}
                          chi.dat.---2.~{}~{}
                          zig.dat.---0.~{}
                          zig.dat.---1.~{}
    
    example.split1.jar    foo.dat.---1.~{}~{}
                          bar.dat.---1.~{}~{}
                          yin.dat.---1.~{}~{}
                          yin.dat.---2.~{}.---1.~{}
                          chi.dat.---1.~{}

MANIFEST (primary jar only)
---------------------------
    Manifest-Version: 1.0
    IzPack-Version: X.X.X
    Created-By: 1.4.2_04-b05 (Sun Microsystems Inc.)
    Built-By: IzPack
    Class-Path: 
    Main-Class: com.izforge.izpack.installer.Installer
    Split-Jar-Secondary-Count: 2
    Split-Entry-Suffix: .---#.~{}

    foo.dat
    Split-Entry-Jar-IDs: 0 1
    Split-Entry-Suffix: .---#.~{}~{}

    bar.dat
    Split-Entry-Jar-IDs: 0 1
    Split-Entry-Suffix: .---#.~{}~{}

    fig.dat
    Split-Entry-Jar-IDs: 0 1
    Split-Entry-Suffix: .---#.~{}~{}

    fig.dat.---2.~{}
    Split-Entry-Jar-IDs: 0 1

    moa.dat
    Split-Entry-Jar-IDs: 0 1 
\end{verbatim}

\end{document}