Add documentation and examples; Add debugging flag; Minor refactors

pull/35/head
Lennart Spitzner 2017-03-30 22:47:00 +02:00
parent 2b30d83fe8
commit 2dd6fe83f5
16 changed files with 210 additions and 45 deletions

View File

@ -16,12 +16,21 @@ of different syntactical constructs into a raw `BriDoc` value.
(technically a `BriDocF` value, we'll explain soon.)
The input of this translation is the syntax tree produced by
GHC/ExactPrint. The ghc api exposes the syntax tree nodes, and
GHC/ExactPrint. The GHC API exposes the syntax tree nodes, and
ExactPrint adds certain annotations (e.g. information about
in-source comments). The main thing that you will be looking
at here is the ghc api documentation, for example
at here is the GHC API documentation, for example
https://downloads.haskell.org/~ghc/8.0.2/docs/html/libraries/ghc-8.0.2/HsDecls.html
Brittany has several flags for dumping intermediate values of the
transformation process; relevant for the "input" syntax tree are the flags
`--dump-ast-unknown` and `--dump-ast-full`, where the latter will print the
whole ast of the input to stderr, the former will only do so for nodes where
brittany falls back on the ghc-exactprint output (i.e. in those cases where
we don't transform, but do a mere copy).
See [this example ast output](output-example-01.md)
(yeah, raw ASTs are ~~annoying large~~ fun!)
## Two examples of the process producing raw BriDoc
1. For example, `Brittany.hs` contains the following code (shortened a bit):
@ -67,6 +76,12 @@ https://downloads.haskell.org/~ghc/8.0.2/docs/html/libraries/ghc-8.0.2/HsDecls.h
What are the exact semantics of the different `doc..` functions?
Why do we need to wrap the `BriDoc` constructors behind those smart-constructor thingies?)
are not explained yet.
In [this example output](output-example-02.md) the BriDoc tree produced in
this fashion is shown for the trivial input `x :: Maybe Int`. Can you spot
the `BDAlt` node that matches the above `docAlt` invocation? (hint: the
node is used twice, so we can see two identical `BDAlt` nodes.)
This leads directly to:
## Size of BriDoc trees, Sharing and Complexity
@ -78,7 +93,7 @@ the same node.
This means the number of nodes in the `BriDoc` value we produces in general is
exponential in the number for syntax nodes of the input.
But we are targeting for linear run-time, right? So what can save us here?
But we are aiming for linear run-time, right? So what can save us here?
You might think: We have sharing! For `let x = 3+3; (x, x)` we only have one
`x` in memory ever. And indeed, we do the same above: `typeDoc1` and `2` are
used in exactly that manner: Both are referenced once in each of the two
@ -114,10 +129,32 @@ So.. we already mentioned "memoization" there, right?
If the `BriDocF` tree is exponential, the transformations will still
do only linear-amount of "selection work" in order to convert into a
linear-sized `BriDoc` tree.
This property is the defining one that motivates the BriDoc
intermediate representation.
Lets have a look at this selection work! We saw at
[the above example](output-example-02.md) how `x :: Maybe Int` had a
non-trivial raw `BriDoc` representation, already with two nested `BDAlt`
nodes and resulting four alternatives. Removing those nodes is the first
step of the `BriDoc` transformation, and we can
[observe the output after removing those nodes](output-example-03.md).
Quite a bit shorter, the tree-printing-algorithm even thinks that it fits
in a single line now.
We will not go into detail about how this "alt-transformation" (the one doing
the "selection work" works and what other transformations follow here.
For this example not much happens; you can see so in the output which you
probably already noticed in the last example.
But for the "alt-transformation" itself, lets at least consider what it does:
We traverse the input BriDoc and whenever a `BDAlt` is encountered, one of the
alternatives is chosen; the other alternatives and the `BDAlt` node itself are
discarded.
The choice is made in such a fashion that, well, the final output does not
contain lines with more than 80 columns but otherwise relatively few newlines.
Magic! (for now at least.)
## BriDocF
The `BriDocF f` type encapsulates the idea that each subnode is wrapped
@ -132,7 +169,8 @@ Lets have a glance at related code/types we have so far:
-- The pure BriDoc: What we really want, but cannot use everywhere due
-- to sharing issues.
-- Isomorphic to `BriDocF Identity`. We still use this type, because
-- then we have to unwrap the `Identities` only in once place.
-- then we have to unwrap the `Identities` only in once place after reducing
-- the tree to a non-exponentially-sized one.
data BriDoc
= BDEmpty
| BDLit !Text
@ -229,4 +267,4 @@ the scenes. For this reason, we will focus on the smart
constructors in the following, because they define the
real interface to be used.
You now might have a glance at "bridoc-api.md"
You now might have a glance at [bridoc-api.md](bridoc-api.md).

View File

@ -0,0 +1,59 @@
invocation:
~~~~
> brittany --dump-ast-full
~~~~
input (via stdin, remember ctrl-d to end-of-file):
~~~~
id :: a -> a
~~~~
output (all but the last line is stderr):
~~~~
---- ast ----
A Just (Ann (DP (0,0)) [] [] [((G AnnEofPos),DP (1,0))] Nothing Nothing)
HsModule
Nothing
Nothing
[]
[ A Just (Ann (DP (0,0)) [] [] [((G AnnDcolon),DP (0,1))] Nothing Nothing)
SigD
TypeSig
[ A Just (Ann (DP (0,0)) [] [] [((G AnnVal),DP (0,0))] Nothing Nothing)
Unqual {OccName: id}
]
HsIB
PlaceHolder
HsWC
PlaceHolder
Nothing
A Just (Ann (DP (0,1)) [] [] [((G AnnRarrow),DP (0,1))] Nothing Nothing)
HsFunTy
A Just (Ann (DP (0,0)) [] [] [] Nothing Nothing)
HsAppsTy
[ A Just (Ann (DP (0,0)) [] [] [] Nothing Nothing)
HsAppPrefix
A Just (Ann (DP (0,0)) [] [] [] Nothing Nothing)
HsTyVar
A Just (Ann (DP (0,0)) [] [] [((G AnnVal),DP (0,0))] Nothing Nothing)
Unqual {OccName: a}
]
A Just (Ann (DP (0,1)) [] [] [] Nothing Nothing)
HsAppsTy
[ A Just (Ann (DP (0,0)) [] [] [] Nothing Nothing)
HsAppPrefix
A Just (Ann (DP (0,0)) [] [] [] Nothing Nothing)
HsTyVar
A Just (Ann (DP (0,0)) [] [] [((G AnnVal),DP (0,0))] Nothing Nothing)
Unqual {OccName: a}
]
]
Nothing
Nothing
----
id :: a -> a
~~~~

View File

@ -0,0 +1,46 @@
invocation:
~~~~
> brittany --dump-bridoc-raw
~~~~
input (via stdin, remember ctrl-d to end-of-file):
~~~~
x :: Maybe Int
~~~~
output (all but the last line is stderr):
~~~~
---- bridoc raw ----
BDAlt
[ BDSeq
[ BDSeq [BDLit (pack "x"),BDSeparator]
, BDSeq [BDLit (pack "::"),BDSeparator]
, BDForceSingleline
BDAlt
[ BDSeq [BDForceSingleline (BDLit (pack "Maybe")),BDLit (pack " "),BDForceSingleline (BDLit (pack "Int"))]
, BDPar BrIndentNone (BDLit (pack "Maybe")) (BDLines [BDEnsureIndent BrIndentRegular (BDLit (pack "Int"))])
]
]
, BDAddBaseY
BrIndentRegular
BDPar
BrIndentNone
BDLit (pack "x")
BDCols
ColTyOpPrefix
[ BDLit (pack ":: ")
, BDAddBaseY
BrIndentSpecial 3
BDAlt
[ BDSeq [BDForceSingleline (BDLit (pack "Maybe")),BDLit (pack " "),BDForceSingleline (BDLit (pack "Int"))]
, BDPar BrIndentNone (BDLit (pack "Maybe")) (BDLines [BDEnsureIndent BrIndentRegular (BDLit (pack "Int"))])
]
]
]
----
x :: Maybe Int
~~~~

View File

@ -0,0 +1,23 @@
invocation:
~~~~
> brittany --dump-bridoc-alt --dump-bridoc-final
~~~~
input (via stdin, remember ctrl-d to end-of-file):
~~~~
x :: Maybe Int
~~~~
output (all but the last line is stderr):
~~~~
---- bridoc post-alt ----
BDSeq [BDSeq [BDLit (pack "x"),BDSeparator],BDSeq [BDLit (pack "::"),BDSeparator],BDSeq [BDLit (pack "Maybe"),BDLit (pack " "),BDLit (pack "Int")]]
---- bridoc final ----
BDSeq [BDLit (pack "x"),BDSeparator,BDLit (pack "::"),BDSeparator,BDLit (pack "Maybe"),BDLit (pack " "),BDLit (pack "Int")]
----
x :: Maybe Int
~~~~

View File

@ -147,7 +147,8 @@ mainCmdParser helpDesc = do
-- let out = do
-- decl <- someDecls
-- ExactPrint.exactPrint decl anns
(errsWarns, outLText) <- if hasCPP
let omitCheck = config & _conf_errorHandling .> _econf_omit_output_valid_check .> confUnpack
(errsWarns, outLText) <- if hasCPP || omitCheck
then return $ pPrintModule config anns parsedSource
else pPrintModuleAndCheck config anns parsedSource
let customErrOrder LayoutWarning{} = 0 :: Int

View File

@ -49,8 +49,7 @@ import qualified Data.Text.Lazy.Builder as Text.Builder
import qualified Language.Haskell.GHC.ExactPrint.Types as ExactPrint
import Language.Haskell.Brittany.Utils
import GHC ( GenLocated(L), moduleNameString )
import SrcLoc ( SrcSpan )
import GHC ( Located, GenLocated(L), moduleNameString )
@ -505,7 +504,7 @@ layoutWritePriorComments
, MonadMultiState LayoutState m
, MonadMultiWriter (Seq String) m
)
=> GenLocated SrcSpan ast
=> Located ast
-> m ()
layoutWritePriorComments ast = do
mAnn <- do
@ -539,7 +538,7 @@ layoutWritePostComments :: (Data.Data.Data ast,
MonadMultiWriter Text.Builder.Builder m,
MonadMultiState LayoutState m
, MonadMultiWriter (Seq String) m)
=> GenLocated SrcSpan ast -> m ()
=> Located ast -> m ()
layoutWritePostComments ast = do
mAnn <- do
state <- mGet
@ -591,7 +590,7 @@ layoutIndentRestorePostComment = do
-- MonadMultiWriter Text.Builder.Builder m,
-- MonadMultiState LayoutState m
-- , MonadMultiWriter (Seq String) m)
-- => GenLocated SrcSpan ast -> m ()
-- => Located ast -> m ()
-- layoutWritePriorCommentsRestore x = do
-- layoutWritePriorComments x
-- layoutIndentRestorePostComment
@ -600,7 +599,7 @@ layoutIndentRestorePostComment = do
-- MonadMultiWriter Text.Builder.Builder m,
-- MonadMultiState LayoutState m
-- , MonadMultiWriter (Seq String) m)
-- => GenLocated SrcSpan ast -> m ()
-- => Located ast -> m ()
-- layoutWritePostCommentsRestore x = do
-- layoutWritePostComments x
-- layoutIndentRestorePostComment

View File

@ -55,6 +55,7 @@ configParser = do
outputOnErrors <- addSimpleBoolFlag "" ["output-on-errors"] (flagHelp $ parDoc "even when there are errors, produce output (or try to to the degree possible")
wError <- addSimpleBoolFlag "" ["werror"] (flagHelp $ parDoc "treat warnings as errors")
omitValidCheck <- addSimpleBoolFlag "" ["omit-output-check"] (flagHelp $ parDoc "omit checking if the output is syntactically valid; for dev on brittany")
optionsGhc <- addFlagStringParams
""
@ -94,6 +95,7 @@ configParser = do
, _econf_Werror = wrapLast $ falseToNothing wError
, _econf_CPPMode = mempty
, _econf_ExactPrintFallback = mempty
, _econf_omit_output_valid_check = wrapLast $ falseToNothing omitValidCheck
}
, _conf_forward = ForwardOptions
{ _options_ghc = [ optionsGhc & List.unwords & CmdArgs.splitArgs | not $ null optionsGhc ]

View File

@ -75,6 +75,7 @@ data ErrorHandlingConfigF f = ErrorHandlingConfig
-- the syntactic validity of the brittany output, at least in theory there
-- may be cases where the output is syntactically/semantically valid but
-- has different semantics that the code pre-transformation.
, _econf_omit_output_valid_check :: f (Semigroup.Last Bool)
}
deriving (Generic)
@ -329,6 +330,7 @@ staticDefaultConfig = Config
, _econf_Werror = coerce False
, _econf_CPPMode = coerce CPPModeAbort
, _econf_ExactPrintFallback = coerce ExactPrintFallbackModeInline
, _econf_omit_output_valid_check = coerce False
}
, _conf_forward = ForwardOptions
{ _options_ghc = Identity []
@ -369,12 +371,13 @@ instance CZip LayoutConfigF where
(f x8 y8)
instance CZip ErrorHandlingConfigF where
cZip f (ErrorHandlingConfig x1 x2 x3 x4)
(ErrorHandlingConfig y1 y2 y3 y4) = ErrorHandlingConfig
cZip f (ErrorHandlingConfig x1 x2 x3 x4 x5)
(ErrorHandlingConfig y1 y2 y3 y4 y5) = ErrorHandlingConfig
(f x1 y1)
(f x2 y2)
(f x3 y3)
(f x4 y4)
(f x5 y5)
instance CZip ForwardOptionsF where
cZip f (ForwardOptions x1)

View File

@ -69,9 +69,8 @@ import Language.Haskell.Brittany.Types
import Language.Haskell.Brittany.Utils
import RdrName ( RdrName(..) )
import GHC ( runGhc, GenLocated(L), moduleNameString )
import GHC ( Located, runGhc, GenLocated(L), moduleNameString )
import qualified SrcLoc as GHC
import SrcLoc ( SrcSpan )
import OccName ( occNameString )
import Name ( getOccString )
import Module ( moduleName )
@ -89,7 +88,7 @@ processDefault
, MonadMultiWriter Text.Builder.Builder m
, MonadMultiReader ExactPrint.Types.Anns m
)
=> GenLocated SrcSpan ast
=> Located ast
-> m ()
processDefault x = do
anns <- mAsk
@ -109,7 +108,7 @@ processDefault x = do
-- syntactic constructs when children are not handled yet.
briDocByExact
:: (ExactPrint.Annotate.Annotate ast)
=> GenLocated SrcSpan ast
=> Located ast
-> ToBriDocM BriDocNumbered
briDocByExact ast = do
anns <- mAsk
@ -125,7 +124,7 @@ briDocByExact ast = do
-- this, e.g. for any top-level declarations.
briDocByExactNoComment
:: (ExactPrint.Annotate.Annotate ast)
=> GenLocated SrcSpan ast
=> Located ast
-> ToBriDocM BriDocNumbered
briDocByExactNoComment ast = do
anns <- mAsk
@ -140,7 +139,7 @@ briDocByExactNoComment ast = do
briDocByExactInlineOnly
:: (ExactPrint.Annotate.Annotate ast, Data ast)
=> String
-> GenLocated SrcSpan ast
-> Located ast
-> ToBriDocM BriDocNumbered
briDocByExactInlineOnly infoStr ast = do
anns <- mAsk
@ -179,7 +178,7 @@ lrdrNameToText (L _ n) = rdrNameToText n
lrdrNameToTextAnn
:: (MonadMultiReader Config m, MonadMultiReader (Map AnnKey Annotation) m)
=> GenLocated SrcSpan RdrName
=> Located RdrName
-> m Text
lrdrNameToTextAnn ast@(L _ n) = do
anns <- mAsk
@ -201,7 +200,7 @@ lrdrNameToTextAnn ast@(L _ n) = do
lrdrNameToTextAnnTypeEqualityIsSpecial
:: (MonadMultiReader Config m, MonadMultiReader (Map AnnKey Annotation) m)
=> GenLocated SrcSpan RdrName
=> Located RdrName
-> m Text
lrdrNameToTextAnnTypeEqualityIsSpecial ast = do
x <- lrdrNameToTextAnn ast
@ -271,7 +270,7 @@ allocNodeIndex = do
-- docLit t = allocateNode $ BDFLit t
--
-- docExt :: (ExactPrint.Annotate.Annotate ast, MonadMultiState NodeAllocIndex m)
-- => GenLocated SrcSpan ast -> ExactPrint.Types.Anns -> Bool -> m BriDocNumbered
-- => Located ast -> ExactPrint.Types.Anns -> Bool -> m BriDocNumbered
-- docExt x anns shouldAddComment = allocateNode $ BDFExternal
-- (ExactPrint.Types.mkAnnKey x)
-- (foldedAnnKeys x)
@ -323,7 +322,7 @@ allocNodeIndex = do
--
--
-- docPostComment :: (Data.Data.Data ast, MonadMultiState NodeAllocIndex m)
-- => GenLocated SrcSpan ast
-- => Located ast
-- -> m BriDocNumbered
-- -> m BriDocNumbered
-- docPostComment ast bdm = do
@ -331,7 +330,7 @@ allocNodeIndex = do
-- allocateNode $ BDFAnnotationPost (ExactPrint.Types.mkAnnKey ast) bd
--
-- docWrapNode :: ( Data.Data.Data ast, MonadMultiState NodeAllocIndex m)
-- => GenLocated SrcSpan ast
-- => Located ast
-- -> m BriDocNumbered
-- -> m BriDocNumbered
-- docWrapNode ast bdm = do
@ -371,7 +370,7 @@ docLit t = allocateNode $ BDFLit t
docExt
:: (ExactPrint.Annotate.Annotate ast)
=> GenLocated SrcSpan ast
=> Located ast
-> ExactPrint.Types.Anns
-> Bool
-> ToBriDocM BriDocNumbered
@ -389,6 +388,7 @@ docAltFilter = docAlt . map snd . filter fst
docSeq :: [ToBriDocM BriDocNumbered] -> ToBriDocM BriDocNumbered
docSeq [] = docEmpty
docSeq l = allocateNode . BDFSeq =<< sequence l
docLines :: [ToBriDocM BriDocNumbered] -> ToBriDocM BriDocNumbered
@ -460,7 +460,7 @@ docParenLSep = appSep $ docLit $ Text.pack "("
docNodeAnnKW
:: Data.Data.Data ast
=> GenLocated SrcSpan ast
=> Located ast
-> Maybe AnnKeywordId
-> ToBriDocM BriDocNumbered
-> ToBriDocM BriDocNumbered
@ -469,15 +469,15 @@ docNodeAnnKW ast kw bdm =
class DocWrapable a where
docWrapNode :: ( Data.Data.Data ast)
=> GenLocated SrcSpan ast
=> Located ast
-> ToBriDocM a
-> ToBriDocM a
docWrapNodePrior :: ( Data.Data.Data ast)
=> GenLocated SrcSpan ast
=> Located ast
-> ToBriDocM a
-> ToBriDocM a
docWrapNodeRest :: ( Data.Data.Data ast)
=> GenLocated SrcSpan ast
=> Located ast
-> ToBriDocM a
-> ToBriDocM a

View File

@ -16,7 +16,6 @@ import Language.Haskell.Brittany.LayouterBasics
import RdrName ( RdrName(..) )
import GHC ( runGhc, GenLocated(L), moduleNameString, AnnKeywordId(..) )
import SrcLoc ( SrcSpan )
import HsSyn
import Name
import qualified FastString

View File

@ -16,7 +16,6 @@ import Language.Haskell.Brittany.LayouterBasics
import RdrName ( RdrName(..) )
import GHC ( runGhc, GenLocated(L), moduleNameString )
import SrcLoc ( SrcSpan )
import HsSyn
import Name

View File

@ -14,8 +14,7 @@ import Language.Haskell.Brittany.Types
import Language.Haskell.Brittany.LayouterBasics
import RdrName ( RdrName(..) )
import GHC ( runGhc, GenLocated(L), moduleNameString )
import SrcLoc ( SrcSpan )
import GHC ( Located, runGhc, GenLocated(L), moduleNameString )
import HsSyn
import Name
import BasicTypes
@ -142,7 +141,7 @@ colsWrapPat :: Seq BriDocNumbered -> ToBriDocM BriDocNumbered
colsWrapPat = docCols ColPatterns . fmap return . Foldable.toList
wrapPatPrepend
:: GenLocated SrcSpan (Pat RdrName)
:: Located (Pat RdrName)
-> ToBriDocM BriDocNumbered
-> ToBriDocM (Seq BriDocNumbered)
wrapPatPrepend pat prepElem = do
@ -154,7 +153,7 @@ wrapPatPrepend pat prepElem = do
return $ x1' Seq.<| xR
wrapPatListy
:: [GenLocated SrcSpan (Pat RdrName)]
:: [Located (Pat RdrName)]
-> String
-> String
-> ToBriDocM (Seq BriDocNumbered)

View File

@ -14,7 +14,6 @@ import Language.Haskell.Brittany.LayouterBasics
import RdrName ( RdrName(..) )
import GHC ( runGhc, GenLocated(L), moduleNameString )
import SrcLoc ( SrcSpan )
import HsSyn
import Name
import qualified FastString

View File

@ -14,7 +14,6 @@ import Language.Haskell.Brittany.LayouterBasics
import RdrName ( RdrName(..) )
import GHC ( runGhc, GenLocated(L), moduleNameString )
import SrcLoc ( SrcSpan )
import HsSyn
import Name
import qualified FastString

View File

@ -16,7 +16,6 @@ import Language.Haskell.Brittany.LayouterBasics
import RdrName ( RdrName(..) )
import GHC ( runGhc, GenLocated(L), moduleNameString )
import Language.Haskell.GHC.ExactPrint.Types ( mkAnnKey )
import SrcLoc ( SrcSpan )
import HsSyn
import Name
import Outputable ( ftext, showSDocUnsafe )

View File

@ -17,8 +17,7 @@ import qualified Language.Haskell.GHC.ExactPrint as ExactPrint
import qualified Data.Text.Lazy.Builder as Text.Builder
import RdrName ( RdrName(..) )
import GHC ( runGhc, GenLocated(L), moduleNameString, AnnKeywordId )
import SrcLoc ( SrcSpan )
import GHC ( Located, runGhc, GenLocated(L), moduleNameString, AnnKeywordId )
import Language.Haskell.GHC.ExactPrint ( AnnKey, Comment )
import Language.Haskell.GHC.ExactPrint.Types ( KeywordId, Anns, DeltaPos, mkAnnKey )
@ -181,9 +180,9 @@ data BrIndent = BrIndentNone
type ToBriDocM = MultiRWSS.MultiRWS '[Config, Anns] '[[LayoutError], Seq String] '[NodeAllocIndex]
type ToBriDoc (sym :: * -> *) = GenLocated SrcSpan (sym RdrName) -> ToBriDocM BriDocNumbered
type ToBriDoc' sym = GenLocated SrcSpan sym -> ToBriDocM BriDocNumbered
type ToBriDocC sym c = GenLocated SrcSpan sym -> ToBriDocM c
type ToBriDoc (sym :: * -> *) = Located (sym RdrName) -> ToBriDocM BriDocNumbered
type ToBriDoc' sym = Located sym -> ToBriDocM BriDocNumbered
type ToBriDocC sym c = Located sym -> ToBriDocM c
data DocMultiLine
= MultiLineNo
@ -324,6 +323,7 @@ instance Uniplate.Uniplate BriDoc where
newtype NodeAllocIndex = NodeAllocIndex Int
-- TODO: rename to "dropLabels" ?
unwrapBriDocNumbered :: BriDocNumbered -> BriDoc
unwrapBriDocNumbered tpl = case snd tpl of
BDFEmpty -> BDEmpty