Add ALDashboard to auto-documentation generation system

.github/workflows/deploy.yml

@@ -38,6 +38,10 @@ jobs:
         with:
           repository: SuffolkLITLab/docassemble-EFSPIntegration
           path: docassemble-EFSPIntegration
+      - uses: actions/checkout@v3
+        with:
+          repository: SuffolkLITLab/docassemble-ALDashboard
+          path: docassemble-ALDashboard
       - name: Go to Docs directory
         run: |
           cd docs

.github/workflows/test-deploy.yml

@@ -38,6 +38,10 @@ jobs:
         with:
           repository: SuffolkLITLab/docassemble-EFSPIntegration
           path: docassemble-EFSPIntegration
+      - uses: actions/checkout@v3
+        with:
+          repository: SuffolkLITLab/docassemble-ALDashboard
+          path: docassemble-ALDashboard
       - name: Go to Docs directory
         run: |
           cd docs

docs/components/ALDashboard/aldashboard.md

@@ -0,0 +1,167 @@
+# Table of Contents
+
+* [ALDashboard.aldashboard](#ALDashboard.aldashboard)
+  * [speedy\_get\_users](#ALDashboard.aldashboard.speedy_get_users)
+  * [speedy\_get\_sessions](#ALDashboard.aldashboard.speedy_get_sessions)
+  * [dashboard\_get\_session\_variables](#ALDashboard.aldashboard.dashboard_get_session_variables)
+  * [ALPackageInstaller](#ALDashboard.aldashboard.ALPackageInstaller)
+    * [get\_validated\_github\_username](#ALDashboard.aldashboard.ALPackageInstaller.get_validated_github_username)
+  * [ErrorList](#ALDashboard.aldashboard.ErrorList)
+  * [ErrorLikeObject](#ALDashboard.aldashboard.ErrorLikeObject)
+  * [install\_fonts](#ALDashboard.aldashboard.install_fonts)
+  * [list\_installed\_fonts](#ALDashboard.aldashboard.list_installed_fonts)
+  * [nicer\_interview\_filename](#ALDashboard.aldashboard.nicer_interview_filename)
+  * [list\_question\_files\_in\_package](#ALDashboard.aldashboard.list_question_files_in_package)
+  * [list\_question\_files\_in\_docassemble\_packages](#ALDashboard.aldashboard.list_question_files_in_docassemble_packages)
+
+---
+sidebar_label: aldashboard
+title: ALDashboard.aldashboard
+---
+
+<a id="ALDashboard.aldashboard.speedy_get_users"></a>
+
+#### speedy\_get\_users()
+
+```python
+def speedy_get_users() -> List[Dict[int, str]]
+```
+
+Return a list of all users in the database. Possibly faster than get_user_list().
+
+<a id="ALDashboard.aldashboard.speedy_get_sessions"></a>
+
+#### speedy\_get\_sessions(user\_id: Optional[int] = None, filename: Optional[str] = None, filter\_step1: bool = True, metadata\_key\_name: str = "metadata")
+
+```python
+def speedy_get_sessions(user_id: Optional[int] = None,
+                        filename: Optional[str] = None,
+                        filter_step1: bool = True,
+                        metadata_key_name: str = "metadata") -> List[Tuple]
+```
+
+Return a list of the most recent 500 sessions, optionally tied to a specific user ID.
+
+Each session is a tuple with named columns:
+filename,
+user_id,
+modtime,
+key
+
+<a id="ALDashboard.aldashboard.dashboard_get_session_variables"></a>
+
+#### dashboard\_get\_session\_variables(session\_id: str, filename: str)
+
+```python
+def dashboard_get_session_variables(session_id: str, filename: str)
+```
+
+Return the variables and steps for a given session ID and YAML filename in serializable dictionary format.
+
+<a id="ALDashboard.aldashboard.ALPackageInstaller"></a>
+
+## ALPackageInstaller Objects
+
+```python
+class ALPackageInstaller(DAObject)
+```
+
+Methods and state for installing AssemblyLine.
+
+<a id="ALDashboard.aldashboard.ALPackageInstaller.get_validated_github_username"></a>
+
+#### get\_validated\_github\_username(access\_token: str)
+
+```python
+def get_validated_github_username(access_token: str)
+```
+
+Given a valid GitHub `access_token`, returns the username associated with it.
+Otherwise, adds one or more errors to the installer.
+
+<a id="ALDashboard.aldashboard.ErrorList"></a>
+
+## ErrorList Objects
+
+```python
+class ErrorList(DAList)
+```
+
+Contains `ErrorLikeObject`s so they can be recognized by docassemble.
+
+<a id="ALDashboard.aldashboard.ErrorLikeObject"></a>
+
+## ErrorLikeObject Objects
+
+```python
+class ErrorLikeObject(DAObject)
+```
+
+An object with a `template_name` that identifies the DALazyTemplate that will
+show its error. It can contain any other attributes so its template can access them
+as needed. DAObject doesn&#x27;t seem to be enough to allow template definition.
+
+<a id="ALDashboard.aldashboard.install_fonts"></a>
+
+#### install\_fonts(the\_font\_files: DAFileList)
+
+```python
+def install_fonts(the_font_files: DAFileList)
+```
+
+Install fonts to the server and restart both supervisor and unoconv.
+
+<a id="ALDashboard.aldashboard.list_installed_fonts"></a>
+
+#### list\_installed\_fonts()
+
+```python
+def list_installed_fonts()
+```
+
+List the fonts installed on the server.
+
+<a id="ALDashboard.aldashboard.nicer_interview_filename"></a>
+
+#### nicer\_interview\_filename(filename: str)
+
+```python
+def nicer_interview_filename(filename: str) -> str
+```
+
+Given a filename like docassemble.playground10ALWeaver:data/questions/assembly_line.yml,
+return a less cluttered name like: playground10ALWeaver:assembly_line
+
+<a id="ALDashboard.aldashboard.list_question_files_in_package"></a>
+
+#### list\_question\_files\_in\_package(package\_name: str)
+
+```python
+def list_question_files_in_package(package_name: str) -> Optional[List[str]]
+```
+
+List all the files in the &#x27;data/questions&#x27; directory of a package.
+
+**Arguments**:
+
+- `package_name` _str_ - The name of the package to list files from.
+
+
+**Returns**:
+
+- `List[str]` - A list of filenames in the &#x27;data/questions&#x27; directory of the package.
+
+<a id="ALDashboard.aldashboard.list_question_files_in_docassemble_packages"></a>
+
+#### list\_question\_files\_in\_docassemble\_packages()
+
+```python
+def list_question_files_in_docassemble_packages()
+```
+
+List all the files in the &#x27;data/questions&#x27; directory of all docassemble packages.
+
+**Returns**:
+
+  Dict[str, List[str]]: A dictionary where the keys are package names and the values are lists of filenames in the &#x27;data/questions&#x27; directory of the package.
+

docs/components/ALDashboard/aldashboard_overview.md

@@ -0,0 +1,34 @@
+---
+id: aldashboard_overview
+title: ALDashboard overview
+sidebar_label: About ALDashboard
+slug: /components/ALDashboard/overview
+---
+
+ALDashboard is a collection of tools to help administer a Docassemble server and debug interviews. It provides utilities for package management, server maintenance, translation workflows, and debugging tools specifically designed for the Document Assembly Line ecosystem.
+
+:::warning This page is a stub
+
+We have not yet documented the key modules in the ALDashboard.
+:::
+
+## Main Modules
+
+ALDashboard consists of several Python modules, each providing specific functionality:
+
+- **aldashboard.py** - Core dashboard functionality and server management
+- **create_package.py** - Package creation and management utilities
+- **translation.py** - Translation workflow and management tools
+- **project_maintenance.py** - Project maintenance and update utilities
+- **docx_wrangling.py** - DOCX document processing and validation
+- **package_scanner.py** - Package analysis and scanning tools
+- **validate_docx.py** - DOCX validation utilities
+- **validate_attachment.py** - Attachment validation tools
+
+Some of this functionality is useful outside of the ALDashboard. `project_maintenance.py` is specifically a tool to run from the commandline and is not available as a "widget" you can interact with from the Docassemble frontend.
+
+## Installation
+
+ALDashboard is typically installed as part of the Document Assembly Line setup:
+
+- [Installation Guide](/docs/get_started/installation)

docs/components/ALDashboard/create_package.md

@@ -0,0 +1,67 @@
+# Table of Contents
+
+* [ALDashboard.create\_package](#ALDashboard.create_package)
+  * [create\_package\_zip](#ALDashboard.create_package.create_package_zip)
+
+---
+sidebar_label: create_package
+title: ALDashboard.create_package
+---
+
+<a id="ALDashboard.create_package.create_package_zip"></a>
+
+#### create\_package\_zip(pkgname: str, info: dict, author\_info: dict, folders\_and\_files: dict, fileobj: Optional[DAFile] = None)
+
+```python
+def create_package_zip(pkgname: str,
+                       info: dict,
+                       author_info: dict,
+                       folders_and_files: dict,
+                       fileobj: Optional[DAFile] = None) -> DAFile
+```
+
+Given a dictionary of lists, with the keys representing folders and the values
+representing a list of DAFiles, create a Python package with Docassemble conventions.
+info: (created by DAInterview.package_info())
+license
+author_name
+readme
+description
+url
+version
+dependencies
+// interview_files replaced with folders_and_files
+// template_files
+// module_files
+// static_files
+author_info:
+author name and email
+folders_and_files:
+questions-&gt;list of absolute file paths on the local filesystem
+templates
+modules
+static
+sources
+
+Strucure of a docassemble package:
++ docassemble-PKGNAME/
+LICENSE
+MANIFEST.in
+README.md
+setup.cfg
+setup.py
++-------docassemble
+__init__.py
++------PKGNAME
+__init__.py
+SOME_MODULE.py
++------data
++------questions
+README.md
++------sources
+README.md
++------static
+README.md
++------templates
+README.md
+

docs/components/ALDashboard/docx_wrangling.md

@@ -0,0 +1,80 @@
+# Table of Contents
+
+* [ALDashboard.docx\_wrangling](#ALDashboard.docx_wrangling)
+  * [update\_docx](#ALDashboard.docx_wrangling.update_docx)
+  * [get\_labeled\_docx\_runs](#ALDashboard.docx_wrangling.get_labeled_docx_runs)
+  * [modify\_docx\_with\_openai\_guesses](#ALDashboard.docx_wrangling.modify_docx_with_openai_guesses)
+
+---
+sidebar_label: docx_wrangling
+title: ALDashboard.docx_wrangling
+---
+
+<a id="ALDashboard.docx_wrangling.update_docx"></a>
+
+#### update\_docx(document: Union[docx.document.Document, str], modified\_runs: List[Tuple[int, int, str, int]])
+
+```python
+def update_docx(
+        document: Union[docx.document.Document, str],
+        modified_runs: List[Tuple[int, int, str,
+                                  int]]) -> docx.document.Document
+```
+
+Update the document with modified runs.
+
+**Arguments**:
+
+- `document` - the docx.Document object, or the path to the DOCX file
+- `modified_runs` - a tuple of paragraph number, run number, the modified text, and
+  a number from -1 to 1 indicating whether a new paragraph should be inserted
+  before or after the current paragraph.
+
+
+**Returns**:
+
+  The modified document.
+
+<a id="ALDashboard.docx_wrangling.get_labeled_docx_runs"></a>
+
+#### get\_labeled\_docx\_runs(docx\_path: str, custom\_people\_names: Optional[Tuple[str, str]] = None, openai\_client: Optional[OpenAI] = None)
+
+```python
+def get_labeled_docx_runs(
+        docx_path: str,
+        custom_people_names: Optional[Tuple[str, str]] = None,
+        openai_client: Optional[OpenAI] = None
+) -> List[Tuple[int, int, str, int]]
+```
+
+Scan the DOCX and return a list of modified text with Jinja2 variable names inserted.
+
+**Arguments**:
+
+- `docx_path` - path to the DOCX file
+- `custom_people_names` - a tuple of custom names and descriptions to use in addition to the default ones. Like: (&quot;clients&quot;, &quot;the person benefiting from the form&quot;)
+
+
+**Returns**:
+
+  A list of tuples, each containing a paragraph number, run number, and the modified text of the run.
+
+<a id="ALDashboard.docx_wrangling.modify_docx_with_openai_guesses"></a>
+
+#### modify\_docx\_with\_openai\_guesses(docx\_path: str)
+
+```python
+def modify_docx_with_openai_guesses(docx_path: str) -> docx.document.Document
+```
+
+Uses OpenAI to guess the variable names for a document and then modifies the document with the guesses.
+
+**Arguments**:
+
+- `docx_path` _str_ - Path to the DOCX file to modify.
+
+
+**Returns**:
+
+- `docx.Document` - The modified document, ready to be saved to the same or a new path
+