FParsec: Creating a Parser Dynamically

In one of the previous posts the following definition for parsing simple Push types was discussed:

let internal pushSimple = choice [
                                   pushSimpleTypes
                                   attempt pushIdentifier
                                   attempt pushOperation
                                   attempt pushLiteral
                                 ]

The most noteworthy entry here is pushSimpleTypes. This parser is created dynamically from internally implemented Push types (INTEGER, FLOAT, etc) and may be dynamically extended if it becomes necessary to parse a new type.

The dynamic parser is built by exposing a delegate type:

// override this delegate to parse extended types
ExtendedTypeParser = delegate of string -> PushTypeBase</pre>

This delegate, implemented by a developer simply takes a string and returns an object of a type, derived from PushTypeBase. More about this type in later posts.

The delegate does not have to be implemented if a type does not provide its own parser. This is the case with CODE, EXEC, and NAME types. When a type is implemented, the delegate is bound to the Parser property of the object that implements the given type. This property may have a value of Unchecked.defaultof<ExtendedTypeParser>, or null in C#.

Otherwise the property returns the actual parsing function. Here is the code in the case of FLOAT type.

    // custom parsing
    static member parse s =
        let result = ref Unchecked.defaultof<float>
        if not (System.Double.TryParse(s, result)) 
        then 
            Unchecked.defaultof<PushTypeBase> 
        else 
            new Float(!result) :> PushTypeBase

    override t.Parser 
        with get() = 
            ExtendedTypeParser(Float.parse)

Given this code, we can now create an FParsec type of parser dynamically:

let internal pushExtended (dlgt : ExtendedTypeParser) = attempt (stringToken >>= (dlgt.Invoke >> createValue))

This parser first parses a string token and then pipes it into a function that is actually a composition of two functions: one is the delegate doing the parsing and converting the token to a value of PushTypeBase derivative type, the result of which is taken and converted into a parser of type PushParser<Push>, where Push is a representation of the simple object plugged into the AST that the parser produces. createValue  has a familiar signature:

    let createValue (value : #PushTypeBase) =
        fun stream ->
            let mutable reply = new Reply<Push>()
            if Unchecked.defaultof<#PushTypeBase> = value
            then 
                reply.Status <- Error
                reply.Error <- messageError("Delegate parser returned null")
            else
                reply.Status <- Ok
                reply.Result <- Value(value)
            reply    

Now for the actual parsers, we first discover all of our extended types and collect them. We then look at the collection and for each of the type gather their Parser properties. This collection of parsers is then filtered and an FParsec type parser is created for each of the entries.

    // takes the map of types, returns the parsers for these types
    let discoverParsers =
        stockTypes.Types
        |> Map.map (fun key value -> 
            value.GetType().GetProperties(BindingFlags.Public ||| BindingFlags.NonPublic ||| BindingFlags.Instance)
            |> Array.find (fun p -> p.PropertyType = typeof<ExtendedTypeParser>))
    

    let createLiteral (s : string) = fst (createPushObject (stockTypes.Types.["LITERAL"].GetType()) [| s.Trim([|'\"'|]) |])

    let internal pushExtended (dlgt : ExtendedTypeParser) = attempt (stringToken >>= (dlgt.Invoke >> createValue))
    let internal pushLiteral = stringLiteral |>> createLiteral >>= createValue
     
    // dynamically create the list of simple type parsers
    // also, filter out types that do not implement a parser
    let pushSimpleTypes =
        let parsers = 
            discoverParsers 
            |> Map.fold(fun lst key value -> value.GetValue(stockTypes.Types.[key], null) :?> ExtendedTypeParser :: lst) List.empty
            |> List.filter(fun callback -> not (callback = Unchecked.defaultof<ExtendedTypeParser>))
            |> List.map(fun callback -> pushExtended callback)
        choice parsers

pushSimpleTypes first gets all of the values of Parser properties of known Push types (internal as well as possibly additional), then casts them to the ExtendedParserTypedelegate type, filters out all those elements of the new list that are not set to anything meaningful (not (callback = Unchecked.defaultof<ExtendedTypeParser>) and creates a list of these delgates. In the third step this list is converted to a list of pushExtended type FParsec parses.

Here is a C# example from the test code of extending the Push engine with a parser that parses URLs. An URL has to start with “http:”

       static UrlPushType UrlParse(string url)
        {
            try
            {
                if (!url.Trim().StartsWith("http://", true, CultureInfo.InvariantCulture))
                {
                    return null;
                }

                Uri uri = new Uri(url);
                if (uri.HostNameType != UriHostNameType.Dns)
                {
                    return null;
                }

                return new UrlPushType(uri);

            }
            catch (Exception)
            {
                return null;                
            }
        }

        public override Type.ExtendedTypeParser Parser
        {
            get 
            {
                return new Type.ExtendedTypeParser(UrlParse);
            }
        }