Better behavior for #31, autoformat lists in Markdown

For tpope's markdown syntax, restored standard autoformat behavior for unordered lists, which is generally pretty good.

Ordered lists had annoying behavior for autoformat, so simply disabling autoformat if detected when insert is entered. This isn't ideal, but it's not clear what else can be done now.

Fixed bug for PFormatToggle, where it wasn't properly disabling Vim's autoformat.

Provided help in README for mapping key to toggle autoformat.

README.markdown

@@ -8,9 +8,6 @@
 ![demo](http://i.imgur.com/0KYl5vU.gif)
 - - -
 
-[Note: demo above needs to be updated with the _recently_ renamed
-commands (see below), but the legacy ones shown still work.]
-
 # Features
 
 The _pencil_ plugin aspires to make Vim as powerful a tool for writers as
@@ -48,7 +45,8 @@ Several reasons have been offered:
   to reach for mouse, track pad, or arrow keys
 * Minimal chording, with many mnemonic-friendly commands
 * Sophisticated capabilities for navigating and manipulating text
-* Highly configurable to suit your needs, with many great plugins available
+* Highly configurable, enabling you to build a workflow that suits your
+  needs, with many great plugins available
 * No proprietary format lock-in
 
 But while such reasons might be sound, they remain scant justification to
@@ -76,13 +74,18 @@ tool in expressing yourself. For more details on vi-style editing, see...
 
 # Installation
 
-_pencil_ is best installed using a Vim package manager, such as Vundle,
-Plug, NeoBundle, or Pathogen.
+_pencil_ is best installed using a Vim package manager, such as
+[Vundle][vnd], [Plug][plg], [NeoBundle][nbn], or [Pathogen][pth].
 
 _For those new to Vim: before installing this plugin, consider getting
 comfortable with the basics of Vim by working through one of the many
 tutorials available._
 
+[vnd]: https://github.com/gmarik/Vundle.vim
+[plg]: https://github.com/junegunn/vim-plug
+[nbn]: https://github.com/Shougo/neobundle.vim
+[pth]: https://github.com/tpope/vim-pathogen
+
 #### Vundle
 
 Add to your `.vimrc` and save:
@@ -143,8 +146,8 @@ git clone https://github.com/reedes/vim-pencil
 
 ## Basic initialization
 
-Though optional, initializing _pencil_ by `FileType` will automatically
-set up your buffers for editing prose.
+Initializing _pencil_ by `FileType` is _optional_, though doing so will
+automatically set up your buffers for editing prose.
 
 Add support for your desired filetypes to your `.vimrc`:
 
@@ -172,10 +175,8 @@ augroup pencil
 augroup END
 ```
 
-For other prose-oriented plugins, consult the [See also](#see-also)
-section below.
-
-For more details, see the advanced initialization section below.
+For a list of other prose-oriented plugins, consult the [See
+also](#see-also) section below.
 
 ## Hard line breaks or soft line wrap?
 
@@ -208,27 +209,27 @@ if auto-detect might suggest soft line wrap.
 You can enable, disable, and toggle _pencil_ as a command:
 
 * `Pencil` - initialize _pencil_ with auto-detect for the current buffer
-* `PencilOff` - removes navigation mappings and restores buffer to global settings
-* `PencilToggle` - if on, turns off; if off, initializes with auto-detect
+* `NoPencil` (or `PencilOff`) - removes navigation mappings and restores buffer to global settings
+* `TogglePencil` (or `PencilToggle`) - if on, turns off; if off, initializes with auto-detect
 
 Because auto-detect might not work as intended, you can invoke a command
 to set the behavior for the current buffer:
 
-* `PencilSoft` - initialize _pencil_ with soft line wrap mode
-* `PencilHard` - initialize _pencil_ with hard line break mode (and Vim’s autoformat)
+* `SoftPencil` (or `PencilSoft`) - initialize _pencil_ with soft line wrap mode
+* `HardPencil` (or `PencilHard`) - initialize _pencil_ with hard line break mode (and Vim’s autoformat)
 
 ## Automatic formatting
 
-_The ‘autoformat’ feature affects *PencilHard* (hard line break) mode
+_The ‘autoformat’ feature affects *HardPencil* (hard line break) mode
 only._
 
-When inserting text while in *PencilHard* mode, Vim’s autoformat feature
+When inserting text while in *HardPencil* mode, Vim’s autoformat feature
 will be enabled by default and can offer many of the same benefits as
 soft line wrap.
 
-One useful exception (aka 'blacklisting'): if used with popular 
-prose-oriented syntax plugins, _pencil_ will **not** enable autoformat when 
-you enter Insert mode from inside a code block or table. (See the 
+One useful exception (aka 'blacklisting'): if used with popular
+prose-oriented syntax plugins, _pencil_ will **not** enable autoformat when
+you enter Insert mode from inside a code block or table. (See the
 advanced section below for more details on the blacklisting feature.)
 
 Where you need to manually enable/disable autoformat for the current
@@ -257,10 +258,17 @@ augroup END
 ...where by default, files of type `text` will use hard line endings, but
 with autoformat disabled.
 
+Optionally, you can map a toggle to control autoformat in your `.vimrc`:
+
+```vim
+noremap <buffer> <silent> <F7> :<C-u>PFormatToggle<cr>
+inoremap <buffer> <silent> <F7> <C-o>:PFormatToggle<cr>
+```
+
 ## Manual formatting
 
-Note that you need not rely on autoformat exclusively and can manually
-reformat paragraphs with standard Vim commands:
+Note that you need not rely on Vim’s autoformat exclusively and can
+manually reformat paragraphs with standard Vim commands:
 
 * `gqip` or `gwip` - format current paragraph
 * `vipJ` - unformat (i.e., join all lines with hard line breaks) in current paragraph
@@ -276,7 +284,7 @@ nnoremap <silent> Q gwip
 
 ## Default textwidth
 
-You can configure the textwidth to be used in **PencilHard** mode when no
+You can configure the textwidth to be used in **HardPencil** mode when no
 textwidth is set globally, locally, or available via modeline. It
 defaults to `74`, but you can change that value in your `.vimrc`:
 
@@ -353,7 +361,7 @@ terminal to support **bold** and _italic_ styles.
 ## Status line indicator
 
 Your status line can reflect the wrap mode for _pencil_ buffers. For
-example, `␍` to represent `PencilHard` (hard line break) mode.  To
+example, `␍` to represent `HardPencil` (hard line break) mode.  To
 configure your status line and ruler, add to your `.vimrc`:
 
 ```vim
@@ -391,20 +399,20 @@ You can override _pencil_ and other configuration settings when initializing:
 ```vim
 augroup pencil
   autocmd!
-  autocmd FileType markdown,mkd call pencil#init() |
-                              \ call litecorrect#init() |
-                              \ setl spell spl=en_us fdl=4 noru nonu nornu |
-                              \ setl fdo+=search
-  autocmd Filetype git,gitsendemail,*commit*,*COMMIT* |
-                              \ call pencil#init({'wrap': 'hard', 'textwidth': 72}) |
-                              \ call litecorrect#init() |
-                              \ setl spell spl=en_us et sw=2 ts=2 noai
-  autocmd Filetype mail         call pencil#init({'wrap': 'hard', 'textwidth': 60}) |
-                              \ call litecorrect#init() |
-                              \ setl spell spl=en_us et sw=2 ts=2 noai nonu nornu
-  autocmd Filetype html,xml     call pencil#init({'wrap': 'soft'}) |
-                              \ call litecorrect#init() |
-                              \ setl spell spl=en_us et sw=2 ts=2
+  autocmd FileType markdown,mkd call pencil#init()
+                            \ | call litecorrect#init()
+                            \ | setl spell spl=en_us fdl=4 noru nonu nornu
+                            \ | setl fdo+=search
+  autocmd Filetype git,gitsendemail,*commit*,*COMMIT*
+                            \   call pencil#init({'wrap': 'hard', 'textwidth': 72})
+                            \ | call litecorrect#init()
+                            \ | setl spell spl=en_us et sw=2 ts=2 noai
+  autocmd Filetype mail         call pencil#init({'wrap': 'hard', 'textwidth': 60})
+                            \ | call litecorrect#init()
+                            \ | setl spell spl=en_us et sw=2 ts=2 noai nonu nornu
+  autocmd Filetype html,xml     call pencil#init({'wrap': 'soft'})
+                            \ | call litecorrect#init()
+                            \ | setl spell spl=en_us et sw=2 ts=2
 augroup END
 ```
 
@@ -413,14 +421,14 @@ statements into a function.
 
 ### Autoformat blacklisting
 
-_The ‘autoformat’ feature affects *PencilHard* (hard line break) mode
+_The ‘autoformat’ feature affects *HardPencil* (hard line break) mode
 only._
 
-When editing formatted text, such as a table or code block, autoformat
-will wreak havoc with the formatting. In these cases you will want to
-temporarily deactivate autoformat, such as with the `PFormatOff` or
-`PFormatToggle` commands described above. However, in most cases, you won’t
-need to do this.
+When editing formatted text, such as a table or code block, Vim’s
+autoformat will wreak havoc with the formatting. In these cases you will
+want to temporarily deactivate autoformat, such as with the `PFormatOff`
+or `PFormatToggle` commands described above. However, in most cases, you
+won’t need to do this.
 
 _pencil_ will detect the syntax highlight group at the cursor position to
 determine whether or not autoformat should be activated.
@@ -430,48 +438,25 @@ mode provided that the syntax highlighting group at the cursor position is
 _not_ in the blacklist. The current blacklist is:
 
 ```vim
-let g:pencil#autoformat_blacklist = [
-      \ 'markdownCode',
-      \ 'markdownH[0-9]',
-      \ 'markdownUrl',
-      \ 'markdownIdDeclaration',
-      \ 'markdownLink',
-      \ 'markdownRule',
-      \ 'markdownHighlight[A-Za-z0-9]+',
-      \ 'mkdCode',
-      \ 'mkdRule',
-      \ 'htmlH[0-9]',
-      \ 'mkdIndentCode',
-      \ 'markdownFencedCodeBlock',
-      \ 'markdownInlineCode',
-      \ 'mmdTable[A-Za-z0-9]*',
-      \ 'txtCode',
-      \ 'rstCodeBlock',
-      \ 'rstDirective',
-      \ 'rstLiteralBlock',
-      \ 'rstSections',
-      \ 'texBeginEndName',
-      \ 'texDelimiter',
-      \ 'texDocType',
-      \ 'texInputFile',
-      \ 'texMath',
-      \ 'texRefZone',
-      \ 'texSection$',
-      \ 'texTitle',
-      \ 'asciidocAttribute',
-      \ 'asciidocList',
-      \ 'asciidocLiteral',
-      \ 'asciidoc[A-Za-z]*Block',
-      \ 'asciidoc[A-Za-z]*Macro',
-      \ 'asciidoc[A-Za-z]*Title',
-      \ ]
+  let g:pencil#autoformat_blacklist = [
+        \ 'markdown(Code|H[0-9]|Url|IdDeclaration|Link|Rule|Highlight[A-Za-z0-9]+)',
+        \ 'markdownOrderedListMarker',
+        \ 'mkd(Code|Rule|Delimiter|Link|ListItem|IndentCode)',
+        \ 'htmlH[0-9]',
+        \ 'markdown(FencedCodeBlock|InlineCode)',
+        \ 'mmdTable[A-Za-z0-9]*',
+        \ 'txtCode',
+        \ 'rst(CodeBlock|Directive|LiteralBlock|Sections)',
+        \ 'tex(BeginEndName|Delimiter|DocType|InputFile|Math|RefZone|Title)',
+        \ 'texSection$',
+        \ 'asciidoc(AttributeList|ListLabel|Literal|SideBar|Source|Sect[0-9])',
+        \ 'asciidoc[A-Za-z]*(Block|Macro|Title)',
+        \ ]
 ```
 
-_WARNING: the implementation of this blacklist may be changing in the
+_WARNING: the implementation of this blacklist will be changing in the
 future._
 
-There’s also a whitelist to override the blacklist. See code for details.
-
 You can override this in your `.vimrc`. Additions to defaults with good
 use cases are welcome.
 

autoload/pencil.vim

@@ -44,7 +44,7 @@ fun! s:detect_wrap_mode() abort
   return s:WRAP_MODE_DEFAULT
 endf
 
-fun! s:imap(preserve_completion, key, icmd)
+fun! s:imap(preserve_completion, key, icmd) abort
   if a:preserve_completion
     exe ":ino <buffer> <silent> <expr> " . a:key . " pumvisible() ? \"" . a:key . "\" : \"" . a:icmd . "\""
   el
@@ -52,7 +52,7 @@ fun! s:imap(preserve_completion, key, icmd)
   en
 endf
 
-fun! s:maybe_enable_autoformat()
+fun! s:maybe_enable_autoformat() abort
   " don't enable autoformat if in a code block or table
   let l:okay_to_enable = 1
   let l:line = line('.')
@@ -92,6 +92,10 @@ fun! s:maybe_enable_autoformat()
     if match(synIDattr(l:sid, 'name'),
             \ g:pencil#autoformat_blacklist_re) >= 0
       let l:okay_to_enable = 0
+      "echohl WarningMsg
+      "echo 'hit blacklist line=' . l:line . ' col=' . l:col .
+      "      \ ' name=' . synIDattr(l:sid, 'name')
+      "echohl NONE
       break
     en
   endfo
@@ -117,12 +121,26 @@ fun! s:maybe_enable_autoformat()
       endfo
     en
   en
+  " disallow enable if start of previous line is in blacklist,
+  " (To avoid problem of autowrap screwing up adding a new item
+  " to a list.)
+  if l:okay_to_enable && l:line > 1
+    let l:prev_stack = synstack(l:line - 1, 1)
+    for l:sid in l:prev_stack
+      if len(l:sid) > 0 &&
+              \ match(synIDattr(l:sid, 'name'),
+              \       g:pencil#autoformat_blacklist_re) >= 0
+        let l:okay_to_enable = 0
+        break
+      en
+    endfo
+  en
   if l:okay_to_enable
     set formatoptions+=a
   en
 endf
 
-fun! pencil#setAutoFormat(af)
+fun! pencil#setAutoFormat(af) abort
   " 1=auto, 0=manual, -1=toggle
   if !exists('b:last_autoformat')
     let b:last_autoformat = 0
@@ -136,7 +154,6 @@ fun! pencil#setAutoFormat(af)
       au InsertEnter <buffer> call s:maybe_enable_autoformat()
       au InsertLeave <buffer> set formatoptions-=a
     aug END
-    let b:last_autoformat = l:nu_af
   el
     sil! au! pencil_autoformat * <buffer>
     if l:nu_af && !l:is_hard
@@ -146,6 +163,7 @@ fun! pencil#setAutoFormat(af)
       return
     en
   en
+  let b:last_autoformat = l:nu_af
 endf
 
 " Create mappings for word processing

plugin/pencil.vim

@@ -58,7 +58,7 @@ if !exists('g:pencil#autoformat')
 en
 
 if !exists('g:pencil#autoformat_blacklist')
-  " by default, pencil does NOT start autoformat if inside any of
+  " do not engage autoformat if cursor is inside any of
   " the following syntax groups
   "
   " markdown* (tpope/vim-markdown)
@@ -68,39 +68,18 @@ if !exists('g:pencil#autoformat_blacklist')
   " txtCode (timcharper/textile.vim)
   " rst*,tex*,asciidoc* (syntax file shipped with vim)
   let g:pencil#autoformat_blacklist = [
-        \ 'markdownCode',
-        \ 'markdownH[0-9]',
-        \ 'markdownUrl',
-        \ 'markdownIdDeclaration',
-        \ 'markdownLink',
-        \ 'markdownRule',
-        \ 'markdownHighlight[A-Za-z0-9]+',
-        \ 'mkdCode',
-        \ 'mkdRule',
+        \ 'markdown(Code|H[0-9]|Url|IdDeclaration|Link|Rule|Highlight[A-Za-z0-9]+)',
+        \ 'markdownOrderedListMarker',
+        \ 'mkd(Code|Rule|Delimiter|Link|ListItem|IndentCode)',
         \ 'htmlH[0-9]',
-        \ 'mkdIndentCode',
-        \ 'markdownFencedCodeBlock',
-        \ 'markdownInlineCode',
+        \ 'markdown(FencedCodeBlock|InlineCode)',
         \ 'mmdTable[A-Za-z0-9]*',
         \ 'txtCode',
-        \ 'rstCodeBlock',
-        \ 'rstDirective',
-        \ 'rstLiteralBlock',
-        \ 'rstSections',
-        \ 'texBeginEndName',
-        \ 'texDelimiter',
-        \ 'texDocType',
-        \ 'texInputFile',
-        \ 'texMath',
-        \ 'texRefZone',
+        \ 'rst(CodeBlock|Directive|LiteralBlock|Sections)',
+        \ 'tex(BeginEndName|Delimiter|DocType|InputFile|Math|RefZone|Title)',
         \ 'texSection$',
-        \ 'texTitle',
-        \ 'asciidocAttribute',
-        \ 'asciidocList',
-        \ 'asciidocLiteral',
-        \ 'asciidoc[A-Za-z]*Block',
-        \ 'asciidoc[A-Za-z]*Macro',
-        \ 'asciidoc[A-Za-z]*Title',
+        \ 'asciidoc(AttributeList|ListLabel|Literal|SideBar|Source|Sect[0-9])',
+        \ 'asciidoc[A-Za-z]*(Block|Macro|Title)',
         \ ]
 en
 let g:pencil#autoformat_blacklist_re =
@@ -112,10 +91,8 @@ if !exists('g:pencil#autoformat_inline_whitelist')
   "
   "'markdownCode' (tpope/vim-markdown)
   let g:pencil#autoformat_inline_whitelist = [
-        \ 'markdownCode',
-        \ 'markdownLink',
-        \ 'asciidocMacro',
-        \ 'asciidocQuotedMonospaced',
+        \ 'markdown(Code|Link)',
+        \ 'asciidoc(AttributeRef|Macro)',
         \ ]
 en
 let g:pencil#autoformat_inline_whitelist_re =
@@ -169,23 +146,23 @@ en
 
 com -nargs=0 Pencil         call pencil#init({'wrap': 'on' })
 com -nargs=0 PencilOff      call pencil#init({'wrap': 'off' })
+com -nargs=0 NoPencil       call pencil#init({'wrap': 'off' })
+com -nargs=0 HardPencil     call pencil#init({'wrap': 'hard'})
 com -nargs=0 PencilHard     call pencil#init({'wrap': 'hard'})
+com -nargs=0 SoftPencil     call pencil#init({'wrap': 'soft'})
 com -nargs=0 PencilSoft     call pencil#init({'wrap': 'soft'})
 com -nargs=0 PencilToggle   call pencil#init({'wrap': 'toggle'})
+com -nargs=0 TogglePencil   call pencil#init({'wrap': 'toggle'})
 com -nargs=0 PFormat        call pencil#setAutoFormat(1)
 com -nargs=0 PFormatOff     call pencil#setAutoFormat(0)
 com -nargs=0 PFormatToggle  call pencil#setAutoFormat(-1)
 
-" NOTE: legacy commands will be disabled by default at some point
+" NOTE: legacy commands will be disabled by default on 31-Dec-15
 if !exists('g:pencil#legacyCommands')
   let g:pencil#legacyCommands = 1
 en
 if g:pencil#legacyCommands
-  com -nargs=0 HardPencil    call pencil#init({'wrap': 'hard'})
-  com -nargs=0 SoftPencil    call pencil#init({'wrap': 'soft'})
   com -nargs=0 DropPencil    call pencil#init({'wrap': 'off' })
-  com -nargs=0 NoPencil      call pencil#init({'wrap': 'off' })
-  com -nargs=0 TogglePencil  call pencil#init({'wrap': 'toggle'})
   com -nargs=0 AutoPencil    call pencil#setAutoFormat(1)
   com -nargs=0 ManualPencil  call pencil#setAutoFormat(0)
   com -nargs=0 ShiftPencil   call pencil#setAutoFormat(-1)