Investigate alternatives to hyperref for Form generation in LaTex

[Brian Lewis]

hyperref does not generate a hierarchical field structure based on dots in the field name. ChatGPT reponse below…

[Brian Lewis]

also hyperref can create pdf form fields; but - it has a problem : Most pdf software interprets a dot in the field name (like Me.Name,Given as representing a hierarchy of fields: name is a subfield of me, and Given is a subfield of Name. But hyrperref does not do this and builds no field hierarchy , instead just creating field names with dots in them. Am I doing something wrong, or missing a workaround? This behaviour causes a lot of incompatibilities!

You're correct that the hyperref package in LaTeX has limitations when it comes to creating hierarchical PDF form fields using dots in the field names. Most PDF viewers and editors, such as Adobe Acrobat, interpret dots in field names as hierarchical separators, but hyperref does not support this natively.

Workaround for Hierarchical PDF Form Fields

To create hierarchical form fields in PDFs generated by LaTeX, you can use the eforms package from the insdljs bundle, which offers more advanced PDF form field capabilities. This package allows you to create hierarchical fields more accurately.

Here’s how you can use eforms to create a hierarchical structure in PDF forms:

1. Install the insdljs Bundle:

   • Ensure you have the insdljs bundle installed in your LaTeX distribution. You can find it on CTAN and follow the installation instructions.

2. Use the eforms Package:

   • Include the eforms package in your LaTeX document and define the form fields using JavaScript to achieve the hierarchical structure.

Here’s an example of how to create hierarchical form fields using eforms:

\documentclass{article}
\usepackage{eforms}
\usepackage{insdljs}

\begin{insDLJS}[example]
var mydoc = this;
mydoc.addField("Me", "text", 0, [72, 720, 144, 742]);
mydoc.addField("Me.Name", "text", 0, [72, 696, 144, 718]);
mydoc.addField("Me.Name.Given", "text", 0, [72, 672, 144, 694]);
\end{insDLJS}

\begin{document}
\begin{Form}

\PushButton[name=pushbtn, onclick={app.alert("Button pressed!")}, width=72, height=36, bordercolor=black, backgroundcolor=gray]{Press me}

\TextField[name=Me, width=3in, height=12pt, bordercolor=black, backgroundcolor=white]{Main Field:}
\vspace{0.5cm}

\TextField[name=Me.Name, width=3in, height=12pt, bordercolor=black, backgroundcolor=white]{Name Field:}
\vspace{0.5cm}

\TextField[name=Me.Name.Given, width=3in, height=12pt, bordercolor=black, backgroundcolor=white]{Given Name Field:}

\end{Form}
\end{document}

‌

Explanation

1. Include eforms and insdljs Packages:

   • Include these packages in your LaTeX preamble to enable advanced form field creation and JavaScript embedding.

2. Define JavaScript to Create Hierarchical Fields:

   • Use JavaScript within the insDLJS environment to define the hierarchical structure of the form fields. The addField method is used to create fields, and you can specify the hierarchy using dot notation.

3. Create Form Fields in the Document:

   • Use the Form, TextField, and other form commands provided by eforms to place the form fields in your document. The field names should match the hierarchy defined in the JavaScript code.

This approach allows you to create PDF form fields with a hierarchical structure that is correctly interpreted by PDF viewers and editors.

Summary

The hyperref package does not support hierarchical form fields natively. Instead, use the eforms package from the insdljs bundle, which provides more advanced capabilities for creating hierarchical PDF form fields. By defining the form fields using embedded JavaScript, you can achieve the desired hierarchical structure and ensure compatibility with PDF viewers.

‌

[Brian Lewis]

Trials with iText show it can prepopulate a form as expected when using an existing Kiribati eSurvey, but fails with a LatEx generated PDF. This is because:
when asked to find a field Survey.SchoolName , it looks first for a field Survey, then a child field of that field named SchoolName. But in the Latex/hyperref version, these fields do not exist, there is only a single field named precisely Survey.SchoolName.

Workaround: if using latex/hyperref:
Instead of naming the fields with dots '.' to indicate hierarchical levels, use a different character e.g. ! .

So the field names look like Survey!SchoolNo,

Since we have to programatically parse these name to construct the hierarchical xfdf anyway, (see #1391) this change would have minor impact.

[Brian Lewis]

feat(pdfsurvey): demonstration of simple working generate/prepopulate of form. Support for form name hierarchy based on ! delimiters, in both generate and upload

Resolves #1391, #1396

→ <<cset 1f0e8c1da142>>

[Brian Lewis]

@ghachey the PR on issue 1391 shows a working example of generate and prepopulating PDF:
- it uses itext, so I’ll leave it to you to decide whether you want these references at this point, or else roll back to develop and I will keep the code privately.
- the template latextestPiped.pdf is used for this simple generation. It uses the ! character to indicate hierarchy, and this now works.
- You can try a round trip:

• do a build_deploy to get the assets
• change your context to kemis

---

go to https://localhost:44301/api/pdfsurvey/generate/KPS001/2024 to generate a form
goto https://localhost:44301/#!/pdfsurvey/upload and load that form back, you should see the school number and name you selected correctly displayed.

[Brian Lewis]

@ghachey after further experimenting i think it could be possible to generate forms entirely programatically using itext. This could prove simpler than LaTex in that we can create larger structures (such as grids) as classes, allow pages contain certain components to be assembled into documents, provide access to lookup tables at time of generation (for row/column headings etc)

For a sample, this class willl make a generic age/class grid:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using iText.Forms;
using iText.Forms.Fields;
using iText.Kernel.Pdf;
using iText.Layout;
using iText.Layout.Element;
using iText.Layout.Properties;
using iText.Kernel.Geom;
using iText.Layout.Renderer;
using Colors = iText.Kernel.Colors;
using Borders = iText.Layout.Borders;


namespace itexttest2
{
    public class GenderedGridmaker
    {
        public string[] Columns;
        public string[] Rows;
        public string Tag;

        public Boolean RowTotals;
        public Boolean ColumnTotals;

        public GenderedGridmaker() { }
        public GenderedGridmaker(string name) { }


        public Table Make()
        {
            // Create a table with )
            Table table = new Table(UnitValue.CreatePercentArray(Columns.Length * 2 + 1))
                          .UseAllAvailableWidth();

            // Empty cell in the top-left corner
            table.AddCell(new Cell());

            // Column headings

            for (int i = 0; i < Columns.Length; i++)
            {
                ////////// Create a text form field for each column heading
                ////////PdfTextFormField field = PdfTextFormField.CreateText(pdfDoc, new Rectangle(0, 0), $"C!0{i}", columnHeadings[i]);
                ////////field.SetValue(columnHeadings[i]);
                ////////field.SetReadOnly(true);
                ////////PdfAcroForm.GetAcroForm(pdfDoc, true).AddField(field);

                // rowspan is 2
                table.AddCell(new Cell(1, 2).Add(new Paragraph(Columns[i])));
            }

            // gender headings row
            // Empty cell in the top-left corner
            table.AddCell(new Cell());

            // Column headings

            for (int i = 0; i < Columns.Length; i++)
            {
                ////////// Create a text form field for each column heading
                ////////PdfTextFormField field = PdfTextFormField.CreateText(pdfDoc, new Rectangle(0, 0), $"C!0{i}", columnHeadings[i]);
                ////////field.SetValue(columnHeadings[i]);
                ////////field.SetReadOnly(true);
                ////////PdfAcroForm.GetAcroForm(pdfDoc, true).AddField(field);

                // rowspan is 1
                table.AddCell(new Cell().Add(new Paragraph("M")));
                table.AddCell(new Cell().Add(new Paragraph("F")));
            }

            // Row headings and data fields

            for (int i = 0; i < Rows.Length; i++)
            {
                // Row heading cell
                ////PdfTextFormField rowField = PdfTextFormField.CreateText(pdfDoc, new Rectangle(0, 0), $"R!0{i}", rowHeadings[i].ToString());
                ////rowField.SetValue(rowHeadings[i].ToString());
                ////rowField.SetReadOnly(true);
                ////PdfAcroForm.GetAcroForm(pdfDoc, true).AddField(rowField);

                table.AddCell(new Cell().Add(new Paragraph(Rows[i].ToString())));

                // Data fields for each row
                for (int j = 0; j < Columns.Length; j++)
                {
                    foreach (string g in new string[] { "M", "F" })
                    {
                        string name = $"{Tag}!D!0{i}!0{j}!{g}";
                        Paragraph pp = new Paragraph();
                        pp.SetNextRenderer(new ChildFieldRenderer(new Cell(), name));
                        table.AddCell(new Cell().Add(pp));
                    }

                }
            }

            // gender totals
            table.AddCell(new Cell(1, 1).Add(new Paragraph("Totals")));

            // Data fields for each row
            for (int j = 0; j < Columns.Length; j++)
            {
                foreach (string g in new string[] { "M", "F" })
                {
                    string name = $"{Tag}!D!T!0{j}!{g}";
                    Paragraph pp = new Paragraph();
                    pp.SetNextRenderer(new TotalFieldRenderer(new Cell(), name));
                    table.AddCell(new Cell().Add(pp));
                }

            }
            // column totals
            // 1st col is rowspan above
            table.AddCell(new Cell(1, 1).Add(new Paragraph("Totals")));
            for (int j = 0; j < Columns.Length; j++)
            {
                string name = $"{Tag}!D!T!0{j}!T";
                Paragraph pp = new Paragraph();
                pp.SetNextRenderer(new TotalFieldRenderer(new Cell(), name));
                table.AddCell(new Cell(1, 2).Add(pp));

            }
            // apply some formatting before returning the table

            return table
                .SetBackgroundColor(Colors.WebColors.GetRGBColor("ivory"))
                .SetBorder(new Borders.SolidBorder(Colors.WebColors.GetRGBColor("lightblue"), 1));
            ;
        }

        // Add the table to the document


        public class ChildFieldRenderer : CellRenderer
        {
            private string _name;

            public ChildFieldRenderer(Cell modelElement, string name) : base(modelElement)
            {
                _name = name;
            }

            public override IRenderer GetNextRenderer()
            {
                return new ChildFieldRenderer((Cell)modelElement, _name);
            }

            public override void Draw(DrawContext drawContext)
            {
                PdfDocument thisDoc = drawContext.GetDocument();
                var pttb = new TextFormFieldBuilder(thisDoc, _name);

                Rectangle rect = GetOccupiedAreaBBox();
                rect = new Rectangle(rect.GetLeft(), rect.GetBottom() - 16, rect.GetWidth(), 20);

                PdfTextFormField dataField = pttb
                    .SetWidgetRectangle(rect)
                    .CreateText();
                PdfAcroForm.GetAcroForm(thisDoc, true).AddField(dataField);

                base.Draw(drawContext);



            }
        }
        public class TotalFieldRenderer : CellRenderer
        {
            private string _name;

            public TotalFieldRenderer(Cell modelElement, string name) : base(modelElement)
            {
                _name = name;
            }

            public override IRenderer GetNextRenderer()
            {
                return new TotalFieldRenderer((Cell)modelElement, _name);
            }

            public override void Draw(DrawContext drawContext)
            {
                PdfDocument thisDoc = drawContext.GetDocument();
                var pttb = new TextFormFieldBuilder(thisDoc, _name);

                Rectangle rect = GetOccupiedAreaBBox();
                rect = new Rectangle(rect.GetLeft(), rect.GetBottom() - 16, rect.GetWidth(), 20);

                PdfTextFormField dataField = pttb
                    .SetWidgetRectangle(rect)
                    .CreateText();
                dataField.SetReadOnly(true);

                PdfAcroForm.GetAcroForm(thisDoc, true).AddField(dataField);

                base.Draw(drawContext);
            }
        }
    }
}

‌

Here’s the result, calling twice:

‌