docs update

README.md

@@ -2,7 +2,7 @@
 
 [![Build Status](https://img.shields.io/travis/markdown-it/markdown-it/master.svg?style=flat)](https://travis-ci.org/markdown-it/markdown-it)
 [![NPM version](https://img.shields.io/npm/v/markdown-it.svg?style=flat)](https://www.npmjs.org/package/markdown-it)
-[![Coverage Status](https://img.shields.io/coveralls/markdown-it/markdown-it/master.svg?style=flat)](https://coveralls.io/r/markdown-it/markdown-it)
+[![Coverage Status](https://img.shields.io/coveralls/markdown-it/markdown-it/master.svg?style=flat)](https://coveralls.io/r/markdown-it/markdown-it?branch=master)
 [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/markdown-it/markdown-it)
 
 > Markdown parser done right. Fast and easy to extend.

docs/README.md

@@ -1,5 +1,10 @@
-This folder contains info mostly for plugins developers.
+This folder contains useful info for plugins developers.
 
 If you just use `markdown-it` in your app, see
 [README](https://github.com/markdown-it/markdown-it#markdown-it) and
 [API docs](https://markdown-it.github.io/markdown-it/).
+
+__Content__:
+
+- [Parser architecture & design principles](architerture.md)
+- [Some guidelines for plugin developpers](development.md)

docs/architecture.md

@@ -82,6 +82,26 @@ Where to search more details about tokens:
 - [Live demo](https://markdown-it.github.io/) - type your text ant click `debug` tab.
 
 
+## Rules
+
+Rules are functions, doing "magick" with parser `state` objects. Each rule is
+registered in one of chain with unique name.
+
+Rules are managed by names via [Ruler](https://markdown-it.github.io/markdown-it/#Ruler) instances and `enable` / `disable` methods in [MarkdownIt](https://markdown-it.github.io/markdown-it/#MarkdownIt).
+
+You can note, that some rules have "validation mode" - in this mode rule does not
+modify token stream, and only search end of token. It's one of important design principle - token stream is "write only" on block & inline parse stages.
+
+Parser is designed to keep rules independent. You can safely disable any, or
+add new one. There are no universal recipes how to create new rules - design of
+distributed state machines with good data isolation is tricky business. But you
+can investigate existing rules & plugins to see possible approaches.
+
+Also, in complex cases you can try to ask for help in tracker. Condition is very
+simple - it should be clear from your ticket, that you studied docs, sources,
+and tryed to do something yourself. We never reject with help to real developpers.
+
+
 ## Parse process
 
 This was mentioned in [Data flow](#data-flow), but let's repeat sequence again:

docs/development.md

@@ -18,7 +18,8 @@ Prior to continue, make sure you've readed:
     block & inline rules are usually faster.
   - Sometime it's enougth to modify renderer only (for example, to add
     header IDs or `target="_blank"` for the links).
-2. Search existing [plugins](https://www.npmjs.org/browse/keyword/markdown-it-plugin)
+2. Search existing
+   [plugins](https://www.npmjs.org/browse/keyword/markdown-it-plugin)
    or [rules](https://github.com/markdown-it/markdown-it/tree/master/lib),
    doing something similar. It can me more simple to modify existing code,
    instead of writing from scratch.
@@ -61,6 +62,24 @@ in more convenient way.
 Righ sequence is to split text to several tokens and add link tokens between.
 Result will be: `text` + `link_open` + `text` + `link_close` + `text`.
 
-See implementations of [linkify](https://github.com/markdown-it/markdown-it/blob/master/lib/rules_core/linkify.js) & [emoji](https://github.com/markdown-it/markdown-it-emoji/blob/master/lib/replace.js) - those do similar things.
+See implementations of [linkify](https://github.com/markdown-it/markdown-it/blob/master/lib/rules_core/linkify.js) & [emoji](https://github.com/markdown-it/markdown-it-emoji/blob/master/lib/replace.js) - those do text token splits.
 
 __Note.__ Don't try to replace text with html markup! That's not secure.
+
+
+#### Why my inline rule is not executed?
+
+Inline parser skips porsion of texts for the best speed. It stops only on [small set of chars](https://github.com/markdown-it/markdown-it/blob/master/lib/rules_inline/text.js), which can be tokens. We did not made this list extendable, also for performance reasons.
+
+If you are absolutely sure, that something important is missed there - create a
+ticket and we will add new charcodes.
+
+
+#### Why do you reject to accept some useful things?
+
+We do markdown parser. It should keep "markdown spirit". Other things should
+be kept separate (in plugins, for example). We have no clear criteria, sorry.
+Probably, you will find useful to read [CommonMark forum](http://talk.commonmark.org/) to understand us better.
+
+Of cause, if you find architecture of this parser interesting for another type
+of markup - you are welcome to reuse it in another project.

lib/index.js

@@ -321,18 +321,22 @@ MarkdownIt.prototype.disable = function (list) {
 
 
 /** chainable
- * MarkdownIt.use(plugin, options)
+ * MarkdownIt.use(plugin, params)
  *
- * Load specified plugin with given options into current parser instance.
+ * Load specified plugin with given params into current parser instance.
+ * It's just a sugar to call `plugin(md, params)` with curring.
  *
  * ##### Example
  *
  * ```javascript
+ * var iterator = require('markdown-it-for-inline');
  * var md = require('markdown-it')()
- *             .use(require('makkdown-it-emoji'));
+ *             .use(iterator, 'foo_replace', 'text', function (tokens, idx) {
+ *               tokens[idx].content = tokens[idx].content.replace(/foo/g, 'bar');
+ *             });
  * ```
  **/
-MarkdownIt.prototype.use = function (plugin /*, options, ... */) {
+MarkdownIt.prototype.use = function (plugin /*, params, ... */) {
   var args = [ this ].concat(Array.prototype.slice.call(arguments, 1));
   plugin.apply(plugin, args);
   return this;
@@ -342,15 +346,17 @@ MarkdownIt.prototype.use = function (plugin /*, options, ... */) {
 /** internal
  * MarkdownIt.parse(src, env) -> Array
  * - src (String): source string
- * - env (Object): enviroment variables
+ * - env (Object): environment sandbox
  *
  * Parse input string and returns list of block tokens (special token type
  * "inline" will contain list of inline tokens). You should not call this
  * method directly, until you write custom renderer (for example, to produce
  * AST).
  *
- * `env` is modified with additional info. For example, with references data.
+ * `env` is used to pass data between "distributed" rules (`{}` by default).
- * Also `env` can be used to pass external info to plugins.
+ * For example, references are parsed in different chains, and need sandbox
+ * to store intermediate results. Can be used to inject data in specific cases.
+ * You will not need it with high probability.
  **/
 MarkdownIt.prototype.parse = function (src, env) {
   var state = new StateCore(this, src, env);
@@ -364,12 +370,13 @@ MarkdownIt.prototype.parse = function (src, env) {
 /**
  * MarkdownIt.render(src [, env]) -> String
  * - src (String): source string
- * - env (Object): optional, enviroment variables
+ * - env (Object): environment sandbox
  *
  * Render markdown string into html. It does all magic for you :).
  *
- * `env` is `{}` by default. It's not used now directly, but you can pass
+ * `env` can be used to inject additional metadata (`{}` by default).
- *  with it any additional data to plugins.
+ * But you will not need it with high probability. See also comment
+ * in [[MarkdownIt.parse]].
  **/
 MarkdownIt.prototype.render = function (src, env) {
   env = env || {};
@@ -381,7 +388,7 @@ MarkdownIt.prototype.render = function (src, env) {
 /** internal
  * MarkdownIt.parseInline(src, env) -> Array
  * - src (String): source string
- * - env (Object): enviroment variables
+ * - env (Object): environment sandbox
  *
  * The same as [[MarkdownIt.parse]] but skip all block rules. It returns the
  * block tokens list with th single `inline` element, containing parsed inline
@@ -400,7 +407,7 @@ MarkdownIt.prototype.parseInline = function (src, env) {
 /**
  * MarkdownIt.renderInline(src [, env]) -> String
  * - src (String): source string
- * - env (Object): optional, enviroment variables
+ * - env (Object): environment sandbox
  *
  * Similar to [[MarkdownIt.render]] but for single paragraph content. Result
  * will NOT be wrapped into `<p>` tags.