diff --git a/README.md b/README.md index 9b23031a1..1f41a7066 100644 --- a/README.md +++ b/README.md @@ -1,41 +1,92 @@ # Repository `docs` -Documentation for Stan language and platform +Repository for the sources and published documentation set, versioned for each Stan minor release. -## GitHub Pages +* The Stan User's Guide - example models and techniques for coding statistical models in Stan and using them to do inference and prediction. +* The Stan Reference Manual - specification for Stan language and core inference algorithms. +* The Stan Functions Reference - functions and distributions built into the Stan language. +* The CmdStan Guide - guide to the reference command-line interface. -This repository uses -[GitHub Pages](https://help.github.com/categories/github-pages-basics) -to serve the -[project pages](https://help.github.com/articles/user-organization-and-project-pages/#project-pages-sites) site -with URL `https://mc-stan.org/docs`. -The publishing strategy is to serve the contents of the directory `docs` on branch `master`. -The `docs` directory contains an empty file named `.nojekyll` so that GitHub will treat the contents -as pre-generated HTML instead of trying to run [jekyll](https://jekyllrb.com). -## Directory Structure +## Repository directory structure -* `src` : directory of source files for Stan and CmdStan guides and reference manuals, each in its own named subdirectory. +* `src` : directory of source files for Stan and CmdStan guides and reference manuals, each in its own named subdirectory: + + `src/cmdstan-guide` - CmdStan Guide + + `src/functions-reference` - Stan Functions Reference + + `src/reference-manual` - Stan Reference Manual + + `src/stan-users-guide` - Stan Users Guide * `docs`: the directory `docs` on branch `master` is the [publishing source](https://help.github.com/articles/configuring-a-publishing-source-for-github-pages/) for the project pages site. Whenever a verified member of the Stan organization pushes to `docs` on branch `master`, GitHub (re)builds and (re)deploys the website. - -* `build.py`: python script which compiles markdown files under `src` to html and pdf and populates the `docs` dir with the generated documentation. - + arg 1: MAJOR Stan version, required (expecting number, should be positive int) - + arg 2: MINOR Stan version, required (expecting number, should be positive int) - + arg 3: name of document (optional - will build entire docset) - + arg 4: output format, either "html" or "pdf" (optional - default builds both html and pdfs) - -* `LICENSE`: licensing terms. +## Documentation toolset + +The documentation source files are written in [Rmarkdown](https://rmarkdown.rstudio.com) +and the RStudio's [bookdown package](https://github.com/rstudio/bookdown) converts these to HTML and pdf. +**Required bookdown version:** 2.23 or higher, cf [the bookdown changelog](https://github.com/rstudio/bookdown/blob/main/NEWS.md#changes-in-bookdown-version-023). +Reported via [issue #380](https://github.com/stan-dev/docs/issues/380). + + +The conversion engine is [Pandoc](https://pandoc.org). It is bundled with RStudio. +To use the build scripts to build the docset, +you might need to [install Pandoc](https://pandoc.org/installing.html) separately. + +To build the pdf version of the docs, you will need to [install LaTeX](https://www.latex-project.org/get/) as well. +The Stan documentation uses the [Lucida fonts](https://www.pctex.com/Lucida_Fonts.html), +which must be [installed manually](https://tex.stackexchange.com/questions/88423/manual-font-installation). + + +## Scripts to build and maintain the docset + +**`build.py`** + +The program `build.py` convert the markdown files under `src` to html and pdf and populates the `docs` dir with the generated documentation. +Requires Python 3.7 or higher, due to call to `subprocess.run`, kwarg `capture_output`. + + 2 required argments: Stan version, expecting 2 positive integer arguments, e.g. `2 28` + + 2 optional arguments: . The document name corresponds to the name of the `src` subdirectory or `all`. The output format is either `html` or `pdf`. + +**Build script examples** -## Generating the static site +* `python build.py 2 28` - creates directory `docs/2_28` as needed; populates it will all generated documentation. +* `python build.py 2 28 functions-reference` - builds both HTML and pdf versions of the Stan functions reference, resulting documents are under `docs/2_28` +* `python build.py 2 28 functions-reference pdf` - builds only the pdf version of the Stan functions reference, resulting document is `docs/2_28/functions-reference_2_28.pdf` +* `python build.py 2 28 all pdf` - builds all pdfs from the Stan documentation set, resulting pdfs are in `docs/2_28`. + -1. Install R packages: bookdown, arm, kableExtra, rstan, bayesplot +**Additional scripts** -2. Install on OS and ensure on PATH: pandoc (version at 2.3 or later), pandoc-citeproc, pdflatex +The release process generates a new documentation set and adds links and redirects across the docset. -3. Run `python build.py ( ())` +* `add_redirects.py` manages the redirects from unversioned links to the latest version. +* `link_to_latest.py` adds the "latest version" link into a docset. + +The Stan Functions Reference contains HTML comments which describe the function signature for all functions. The script `extract_function_sigs.py` is used to scrape these signatures into a plain text file. + +## Build a single docset in R: `bookdown::render_book` + +To build a single documet, you must have R or RStudio installed. +To build a document from the command line, first `cd` to the correct `src` dir, +then use the `Rscript` utility. + +``` +# build html +> Rscript -e "bookdown::render_book('index.Rmd', output_format='bookdown::gitbook')" + +# build pdf +> Rscript -e "bookdown::render_book('index.Rmd', output_format='bookdown::pdf_book')" +``` + +The output will be written to subdirectory `_build`. + +## GitHub Pages + +This repository uses +[GitHub Pages](https://help.github.com/categories/github-pages-basics) +to serve the +[project pages](https://help.github.com/articles/user-organization-and-project-pages/#project-pages-sites) site +with URL `https://mc-stan.org/docs`. +The publishing strategy is to serve the contents of the directory `docs` on branch `master`. +The `docs` directory contains an empty file named `.nojekyll` so that GitHub will treat the contents +as pre-generated HTML instead of trying to run [jekyll](https://jekyllrb.com). -4. The generated html and/or pdfs will be in the versioned subdirectory of `docs`. diff --git a/build.py b/build.py index ee94606ed..da2a5782a 100644 --- a/build.py +++ b/build.py @@ -29,11 +29,18 @@ def pushd(new_dir): os.chdir(previous_dir) def shexec(command): - returncode = subprocess.call(command, shell=True) + returncode = subprocess.call(command, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) if returncode != 0: print('Command {} failed'.format(command)) raise Exception(returncode) +def check_bookdown_2_23(): + command = ["Rscript", "-e", "utils::packageVersion(\"bookdown\") > 0.22"] + check = subprocess.run(command, capture_output=True, text=True) + if not 'TRUE' in check.stdout: + print("R Package bookdown must be version 2.23, see README for build tools requirements") + sys.exit(1) + def safe_rm(fname): if os.path.exists(fname): os.remove(fname) @@ -43,13 +50,16 @@ def make_docs(docspath, version, document, formats): srcpath = os.path.join(path, "src", document) with pushd(srcpath): if ("pdf" in formats): + print('render {} as pdf'.format(document)) command = "Rscript -e \"bookdown::render_book(\'index.Rmd\', output_format=\'bookdown::pdf_book\')\"" shexec(command) srcpdf = os.path.join(srcpath, "_book", "_main.pdf") pdfname = ''.join([document,'-',version,".pdf"]) pdfpath = os.path.join(docspath, pdfname) shutil.move(srcpdf, pdfpath) + print('output file: {}'.format(pdfpath)) if ("html" in formats): + print('render {} as html'.format(document)) command = "Rscript -e \"bookdown::render_book(\'index.Rmd\', output_format=\'bookdown::gitbook\')\"" shexec(command) [safe_rm(f) for f in ("stan-manual.css", "_main.rds")] @@ -58,11 +68,16 @@ def make_docs(docspath, version, document, formats): shutil.rmtree(htmlpath, ignore_errors=True) command = ' '.join(["mv _book", htmlpath]) shexec(command) + print('output dir: {}'.format(htmlpath)) else: [safe_rm(f) for f in ("stan-manual.css", "_main.rds")] shutil.rmtree("_book", ignore_errors=True) def main(): + if sys.version_info < (3, 7): + print('requires Python 3.7 or higher, found {}'.format(sys.version)) + sys.exit(1) + check_bookdown_2_23() global all_docs global all_formats if (len(sys.argv) > 2): diff --git a/src/cmdstan-guide/_output.yml b/src/cmdstan-guide/_output.yml index a11d456d2..851451edd 100644 --- a/src/cmdstan-guide/_output.yml +++ b/src/cmdstan-guide/_output.yml @@ -3,7 +3,8 @@ bookdown::gitbook: css: stan-manual.css split_by: "chapter" config: - search: TRUE + search: + engine: lunr sharing: null download: null edit: null diff --git a/src/functions-reference/_output.yml b/src/functions-reference/_output.yml index 290360f5d..55d9f599a 100644 --- a/src/functions-reference/_output.yml +++ b/src/functions-reference/_output.yml @@ -2,7 +2,8 @@ bookdown::gitbook: css: ../stan-manual.css split_by: "section" config: - search: TRUE + search: + engine: lunr sharing: null download: null edit: null @@ -18,10 +19,10 @@ bookdown::gitbook: before: |
  • Stan Functions Reference
    • after: null - subparagraph: yes bookdown::pdf_book: latex_engine: "pdflatex" includes: in_header: header.tex after_body: postamble.tex + diff --git a/src/reference-manual/_output.yml b/src/reference-manual/_output.yml index 927efff19..f8561c4b8 100644 --- a/src/reference-manual/_output.yml +++ b/src/reference-manual/_output.yml @@ -2,7 +2,8 @@ bookdown::gitbook: css: ../stan-manual.css split_by: "section" config: - search: TRUE + search: + engine: lunr sharing: null download: null edit: null diff --git a/src/stan-users-guide/_output.yml b/src/stan-users-guide/_output.yml index 6bd717dda..2e64efbea 100644 --- a/src/stan-users-guide/_output.yml +++ b/src/stan-users-guide/_output.yml @@ -3,7 +3,8 @@ bookdown::gitbook: css: stan-manual.css split_by: "section" config: - search: TRUE + search: + engine: lunr sharing: null download: null edit: null diff --git a/src/stan-users-guide/before-chapter.R b/src/stan-users-guide/before-chapter.R index 7522b0adc..c7f862f00 100644 --- a/src/stan-users-guide/before-chapter.R +++ b/src/stan-users-guide/before-chapter.R @@ -1,11 +1,9 @@ # LIBRARY CONFIGURATION -library(arm) -library(bayesplot) - # theme_set(bayesplot::theme_default()) library(dplyr) library(ggplot2) library(kableExtra) + library(knitr) knitr::opts_chunk$set( cache = TRUE, @@ -24,31 +22,9 @@ library(knitr) tidy = FALSE, warning = FALSE ) -library(lubridate) -library(rstan) - rstan_options(auto_write = FALSE) - options(mc.cores = 1) - - -# UTILITY FUNCTIONS - -printf <- function(pattern, ...) { - cat(sprintf(pattern, ...)) -} - -print_file <- function(file) { - cat(paste(readLines(file), "\n", sep=""), sep="") -} - -extract_one_draw <- function(stanfit, chain = 1, iter = 1) { - x <- get_inits(stanfit, iter = iter) - x[[chain]] -} # GENERAL R CONFIGURATION options(digits = 2) options(htmltools.dir.version = FALSE) - -set.seed(1123) diff --git a/src/stan-users-guide/latent-discrete.Rmd b/src/stan-users-guide/latent-discrete.Rmd index 80701e2c6..94a5c63de 100644 --- a/src/stan-users-guide/latent-discrete.Rmd +++ b/src/stan-users-guide/latent-discrete.Rmd @@ -60,27 +60,27 @@ c \; ? \; x_1 \: : \: x_2 \ x_2 & \quad\text{if } c \text{ is false (i.e., zero).} \end{cases} $$ - ### Marginalizing out the discrete parameter {-} To code this model in Stan, the discrete parameter $s$ must be marginalized out to produce a model defining the log of the -probability function $p(e,l,D_t)$. The full joint probability factors -as +probability function $p(e,l,D_t)$. +The full joint probability factors as \begin{align*} p(e,l,s,D) &= p(e) \, p(l) \, p(s) \, p(D \mid s, e, l) \\ - &= \textsf{exponential}(e \mid r_e) \ \textsf{exponential}(l \mid r_l) \ + &= \textsf{exponential}(e \mid r_e) \ \textsf{exponential}(l \mid r_l) \, \textsf{uniform}(s \mid 1, T) \\ - &\qquad \prod_{t=1}^T \textsf{Poisson}(D_t \mid t < s \; ? \; e \: : \: l), + & \qquad \prod_{t=1}^T \textsf{Poisson}(D_t \mid t < s \; ? \; e \: : \: l). \end{align*} -To marginalize, an alternative factorization into prior and likelihood -is used, + +To marginalize, an alternative factorization into prior and likelihood is used, $$ p(e,l,D) = p(e,l) \, p(D \mid e,l), $$ + where the likelihood is defined by marginalizing $s$ as \begin{align*} p(D \mid e,l) &= \sum_{s=1}^T p(s, D \mid e,l) \\