Fix interaction with "package environment files" (.ghc.environment.*) #173

New Issue

2018-08-19T16:57:00+02:00

lspitzner commented

2018-08-19 16:57:00 +02:00

(Migrated from github.com)

First, the facts:

Since ghc-8.0, ghc considers package environment files that may make packages available for usage (or something in that direction. It is not even relevant what exactly their purpose is). See the ghc users guide
ghc looks for these files in the current working directory and its parents.
cabal has started writing such files automatically, as a side-effect of invoking e.g. cabal new-build.
These files can go stale, and ghc may stop working in such cases.
Even though brittany does not invoke ghc as a process but uses the GHC API, the behaviour is identical, meaning that brittany can randomly stop working in the presence of .ghc.environment.* files.

> cat ./.ghc.environment.x86_64-linux-8.2.2
outdated
> echo "" | brittany
<command line>: cannot satisfy -package-id outdated
(use -v for more information)
Since version 8.6 ghc prints a note about the environment files it considers to stdout/stderr (probably the latter, but I can't be bothered to test). This also affects usages of the GHC API. Consequently, brittany on ghc-8.6+ also prints stuff to stdout/stderr, even if this "feature" does not break the GHC API. This affects both the brittany executable and the brittany library interfaces (used by HIE).

I have opened a ticket in the ghc tracker and pointed out that this behaviour breaks stuff in a relevant cabal issue but so far there is no resolution in sight.

This raises the question of how to best mitigate this (let's be really nice and call it) excessive carelessness for the users of brittany. Let's consider possible solutions:

Call the GHC API in such a way that it does not read environment files. ~~Problem with this approach is that it does not work, according to what I have tested so far.~~ Correction: It works, if you know the magic GHC API words.

2) Write out a clean .ghc.environment file (to CWD) before invoking the GHC API. Downside is that we may overwrite a hand-written file, which the user might not be too happy about.
3) Delete any .ghc.environment files we can find in CWD and parent directories. Same downside as above.
3b) Delete only those generated by cabal; warn otherwise.
4) Detect the existence of .ghc.environment files, and print a nice fat warning. Make this configurable (turn off) via config.
5) Detect only exactly the broken case (bad environment file causes GHC API to fail) and print a note to the user. Downside is that is annoying to implement and does not even cover the brittany API.
6) Accept the behaviour, but document it in the README and other brittany docs.

None of these are satisfactory. Any other ideas welcome. I will have to/want to make a decision before the ghc-8.6 compat release, currently leaning towards 4 together with 6 to cover the brittany API.

First, the facts: 1) Since ghc-8.0, ghc considers package environment files that may make packages available for usage (or something in that direction. It is not even relevant what exactly their purpose is). See [the ghc users guide](https://downloads.haskell.org/~ghc/8.4.3/docs/html/users_guide/packages.html#package-environments) 2) ghc looks for these files in the current working directory and its parents. 3) cabal has started writing such files automatically, as a side-effect of invoking e.g. `cabal new-build`. 4) These files can go stale, and ghc may stop working in such cases. 5) Even though brittany does not invoke `ghc` as a process but uses the GHC API, the behaviour is identical, meaning that `brittany` can randomly stop working in the presence of `.ghc.environment.*` files. > \> cat ./.ghc.environment.x86_64-linux-8.2.2 > outdated > \> echo "" | brittany > \<command line>: cannot satisfy -package-id outdated > (use -v for more information) 6) Since version 8.6 ghc prints a note about the environment files it considers to stdout/stderr (probably the latter, but I can't be bothered to test). This _also_ affects usages of the GHC API. Consequently, brittany on ghc-8.6+ also prints stuff to stdout/stderr, even if this "feature" does not break the GHC API. This affects both the brittany executable and the brittany library interfaces (used by HIE). I have opened a [ticket in the ghc tracker](https://ghc.haskell.org/trac/ghc/ticket/15513) and [pointed out that this behaviour breaks stuff in a relevant cabal issue](https://github.com/haskell/cabal/issues/4542#issuecomment-413374680) but so far there is no resolution in sight. This raises the question of how to best mitigate this (let's be really nice and call it) excessive carelessness for the users of brittany. Let's consider possible solutions: 1) Call the GHC API in such a way that it does not read environment files. ~~Problem with this approach is that it does not work, according to what I have tested so far.~~ Correction: It works, if you know the magic GHC API words. ~~2) Write out a clean `.ghc.environment` file (to CWD) before invoking the GHC API. Downside is that we may overwrite a hand-written file, which the user might not be too happy about. 3) Delete any `.ghc.environment` files we can find in CWD and parent directories. Same downside as above. 3b) Delete only those generated by cabal; warn otherwise. 4) Detect the existence of `.ghc.environment` files, and print a nice fat warning. Make this configurable (turn off) via config. 5) Detect only exactly the broken case (bad environment file causes GHC API to fail) and print a note to the user. Downside is that is annoying to implement and does not even cover the brittany API. 6) Accept the behaviour, but document it in the README and other brittany docs.~~ ~~None of these are satisfactory. Any other ideas welcome. I will have to/want to make a decision before the ghc-8.6 compat release, currently leaning towards 4 together with 6 to cover the brittany API.~~

👍 2

ElvishJerricco commented

2018-08-19 21:25:20 +02:00

(Migrated from github.com)

There must be some way to achieve solution 1, right? GHC has flags that disable the use of these files, so surely brittany can invoke the same behavior?

lspitzner commented

2018-08-19 22:55:59 +02:00

(Migrated from github.com)

@ElvishJerricco As it turns out, yes. The trick is to use this

  dflags0           <- GHC.getSessionDynFlags
  (dflags1, [], []) <- GHC.parseDynamicFlagsCmdLine dflags0 [GHC.noLoc $ "-hide-all-packages"]
  void $ GHC.setSessionDynFlags dflags1

and this needs to be before any other invocation of setSessionDynFlags. I'll make a PR for ghc-exactprint that does this, which hopefully will solve this, for all supported ghc versions.

@ElvishJerricco As it turns out, yes. The trick is to use this ~~~~.hs dflags0 <- GHC.getSessionDynFlags (dflags1, [], []) <- GHC.parseDynamicFlagsCmdLine dflags0 [GHC.noLoc $ "-hide-all-packages"] void $ GHC.setSessionDynFlags dflags1 ~~~~ and this needs to be before any other invocation of `setSessionDynFlags`. I'll make a PR for ghc-exactprint that does this, which hopefully will solve this, for all supported ghc versions.

👍 2

lspitzner commented

2018-08-20 21:21:45 +02:00

(Migrated from github.com)

PR at https://github.com/alanz/ghc-exactprint/pull/68.

Unfortunately brittany also calls setSessionDynFlags before initDynFlags, 33a4039751/src/Language/Haskell/Brittany/Internal/ExactPrintUtils.hs (L70-L77)

And was done on purpose, see 1b7576dcd1. Fixed in 8c5cce5070.

Leaving this open until the changes have landed upstream and the fix is confirmed.

PR at https://github.com/alanz/ghc-exactprint/pull/68. Unfortunately brittany also calls `setSessionDynFlags` before `initDynFlags`, https://github.com/lspitzner/brittany/blob/33a403975169a7ecf8f12121f9788ed8c2a65715/src/Language/Haskell/Brittany/Internal/ExactPrintUtils.hs#L70-L77 And was done on purpose, see 1b7576dcd1823e1c685a44927b1fcaade1319063. Fixed in 8c5cce50709d6c0e41aed13df808a36ce13df4a4. Leaving this open until the changes have landed upstream and the fix is confirmed.

tfausak commented

2019-06-18 14:08:17 +02:00

(Migrated from github.com)

Closing since this is fixed now.

Sign in to join this conversation.

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: hexagoxel/brittany#173