Source

HH-Parse / profile.lisp

Full commit
;; Note this file is not presently used by the package, but is included for 

(sb-profile:profile "HH-PARSE")

(sb-profile:reset)

;; TODO: Need to rethink how we profile, since grammars are now constructed at load time,
;; and that was the most expensive part of this code
(let* ((grammar (html-grammar)) 
       (source (make-instance 'source-code-file))
       (lexer (make-instance 'html-lexer :source source))
       (parser (make-parser lexer grammar))) 
  (setf (source-text source) "<foo bar=1>borp whaple</foo>")
  (parse-token parser (next-token lexer)))

(sb-profile:report)

(sb-profile:unprofile)

#|
LATEST : measuring PROFILE overhead..done
  seconds  |     gc     |    consed   |    calls   |  sec/call  |  name  
--------------------------------------------------------------
     5.922 |      0.005 |  67,385,104 | 17,097,550 |  0.0000003 | EQUAL-ITEMS
     0.672 |      0.016 |  19,591,280 |  1,254,854 |   0.000001 | NEXT-TERM-AND-LOOKAHEAD-FOR-ITEM
     0.616 |      0.158 | 232,306,832 |    438,723 |   0.000001 | RULE-PRODUCTIONS
     0.465 |      0.000 |   5,782,080 |    973,502 |  0.0000005 | RULE-PRODUCTIONS-IN-GRAMMAR
     0.460 |      0.000 |           0 |  1,286,874 |  0.0000004 | ITEM-FINISHED-P
     0.133 |      0.000 |  22,047,408 |    248,857 |   0.000001 | INITIAL-LR1-ITEM-FOR-PRODUCTION
     0.124 |      0.000 |           0 |    224,836 |   0.000001 | ITEM<
     0.075 |      0.000 |           0 |    371,931 |  0.0000002 | ITEMS
     0.028 |      0.000 |           0 |     92,097 |  0.0000003 | EXITS
     0.023 |      0.005 |   2,163,488 |     27,964 |   0.000001 | INCREMENT-LR1-ITEM-POSITION
     0.015 |      0.000 |   1,686,640 |          1 |   0.014592 | (SETF SOURCE-TEXT)
     0.008 |      0.000 |           0 |     20,048 |  0.0000004 | ENTRIES
     0.006 |      0.000 |     618,736 |         29 |   0.000205 | INSERT-INTO
     0.005 |      0.000 |     553,856 |          1 |   0.004993 | MAKE-PARSER
     0.003 |      0.000 |     490,048 |         28 |   0.000093 | INSERT-CHARACTER
     0.001 |      0.000 |       4,080 |          1 |   0.000945 | NEXT-LEX-CHARACTER
     0.001 |      0.000 |       4,064 |         29 |   0.000032 | (SETF COLUMN-AT)
     0.001 |      0.000 |      16,352 |        172 |   0.000004 | CURRENT-LINE-POSITION
    0.0005 |      0.000 |           0 |         28 |   0.000016 | FLUSH-PARSERS
    0.0002 |      0.000 |       8,192 |         56 |   0.000003 | CURRENT-LINE
     0.000 |      0.000 |           0 |          1 |   0.000000 | GRAMMAR
     0.000 |      0.000 |           0 |          1 |   0.000000 | NEXT-TOKEN
     0.000 |      0.000 |           0 |          1 |   0.000000 | NON-TERMINALS-IN-GRAMMAR
     0.000 |      0.000 |  12,137,360 |          1 |   0.000000 | LR1-STATES-FOR-GRAMMAR
     0.000 |      0.000 |           0 |          1 |   0.000000 | CURRENT-LEX-CHARACTER
     0.000 |      0.000 |           0 |          1 |   0.000000 | HTML-GRAMMAR
     0.000 |      0.000 |      77,824 |      4,056 |   0.000000 | NEXT-TERM-FOR-ITEM
     0.000 |      0.000 |       4,080 |          1 |   0.000000 | INITIAL-LR1-STATE-FOR-GRAMMAR
     0.000 |      0.000 |           0 |          1 |   0.000000 | PARSE-TOKEN
     0.000 |      0.027 |  39,510,560 |    832,792 |   0.000000 | FIRST-OF-SYMBOL
     0.000 |      0.000 |           0 |         58 |   0.000000 | COLUMN-AT
     0.000 |      0.000 |           0 |         30 |   0.000000 | TEXT-OF
     0.000 |      0.000 |           0 |         28 |   0.000000 | (SETF TEXT-OF)
     0.000 |      0.000 |       4,080 |          1 |   0.000000 | ACTIONS
     0.000 |      0.000 |   3,730,176 |          1 |   0.000000 | MAKE-GRAMMAR
     0.000 |      0.005 |  21,696,944 |     61,716 |   0.000000 | LR1-GOTO-FOR-SYMBOL
     0.000 |      0.000 |           0 |        117 |   0.000000 | LINE-AT
     0.000 |      0.000 |       4,096 |          2 |   0.000000 | MAKE-LINES
     0.000 |      0.000 |           0 |         27 |   0.000000 | (SETF PARSER-OF)
     0.000 |      0.000 |           0 |          1 |   0.000000 | INCF-LEX-POSITION
     0.000 |      0.000 |           0 |          1 |   0.000000 | PRODUCTIONS-IN-GRAMMAR
     0.000 |      0.079 | 220,378,512 |     61,717 |   0.000000 | CLOSURE-OF-LR1-ITEMS
     0.000 |      0.000 |           0 |          4 |   0.000000 | STACK
     0.000 |      0.000 |       4,096 |          2 |   0.000000 | (SETF STACK)
     0.000 |      0.000 |       4,064 |         56 |   0.000000 | NEXT-COLUMN
     0.000 |      0.000 |           0 |          1 |   0.000000 | SYMBOLS-IN-GRAMMAR
     0.000 |      0.000 |           0 |        169 |   0.000000 | CURRENT-POSITION
     0.000 |      0.000 |           0 |        143 |   0.000000 | LINES-OF
     0.000 |      0.000 |           0 |          1 |   0.000000 | (SETF LINES-OF)
     0.000 |      0.000 |           0 |          1 |   0.000000 | INSERT-TEXT
     0.000 |      0.000 |       8,176 |          2 |   0.000000 | CURRENT-COLUMN-POSITION
     0.000 |      0.000 |           0 |          1 |   0.000000 | (SETF CURRENT-COLUMN-POSITION)
--------------------------------------------------------------
     8.555 |      0.295 | 650,218,128 | 22,998,516 |            | Total

estimated total profiling overhead: 108.78 seconds
overhead estimation parameters:
  8.000001e-9s/call, 4.73e-6s total profiling, 2.216e-6s internal profiling

These functions were not called:
 (SETF ACTIONS) COMPOUNDTERMP COPY-LEXER (SETF CURRENT-LINE)
 (SETF CURRENT-LINE-POSITION) (SETF CURRENT-POSITION) (SETF ENTRIES)
 EQUAL-CORES (SETF EXITS) FOR-EACH-RHS FOR-EACH-RULE GET-PARSE-RESULT
 (SETF GOTOS) GOTOS (SETF GRAMMAR) GRAMMAR-419 GRAMMAR-420 INSERT-LINE
 (SETF ITEMS) (SETF LEXER) LEXER (SETF LINE-AT) MAKE-NEW-NAMED-RULE
 NEXT-LINE (SETF NUMBER-OF-STATES) NUMBER-OF-STATES NUMBERS-GRAMMAR
 PARSE-INPUT (SETF PARSE-POSITION) PARSE-POSITION PARSER-OF POSITIONS-EQUAL
 REPARSE (SETF SOURCE) SOURCE SOURCE-TEXT (SETF SPECIFICATION)
 SPECIFICATION SPLIT-LINE (SETF STATE) STATE (SETF STATES) STATES
 TERMINALS-IN-GRAMMAR TRANSFORM-EXTENDED-GRAMMAR-TO-FUNDAMENTAL
 TRANSFORM-FOR-NIL TRANSFORM-RHS TRANSFORM-TERM (SETF UNEXPECTED-CHARACTER)
 UNEXPECTED-CHARACTER

ORIGINAL : With states as just lists of items:

measuring PROFILE overhead..done
  seconds  |     gc     |     consed    |    calls    |  sec/call  |  name  
-----------------------------------------------------------------
   130.061 |      0.345 |   901,604,240 | 244,373,918 |   0.000001 | EQUAL-ITEMS
     9.573 |      0.013 |    81,127,376 |  12,662,419 |   0.000001 | RULE-PRODUCTIONS-IN-GRAMMAR
     8.690 |      1.041 | 3,168,389,728 |   5,787,254 |   0.000002 | RULE-PRODUCTIONS
     8.141 |      0.047 |   143,667,408 |   9,230,162 |   0.000001 | NEXT-TERM-AND-LOOKAHEAD-FOR-ITEM
     3.850 |      0.000 |             0 |   9,502,265 |  0.0000004 | ITEM-FINISHED-P
     2.107 |      0.000 |             0 |   2,634,401 |   0.000001 | ITEM<
     1.859 |      0.045 |   263,900,208 |   3,031,051 |   0.000001 | INITIAL-LR1-ITEM-FOR-PRODUCTION
     0.183 |      0.007 |    21,801,936 |     268,047 |   0.000001 | INCREMENT-LR1-ITEM-POSITION
     0.015 |      0.000 |             0 |      20,048 |   0.000001 | ENTRIES
     0.008 |      0.000 |        77,824 |       4,056 |   0.000002 | NEXT-TERM-FOR-ITEM
     0.003 |      0.000 |         4,096 |          56 |   0.000057 | CURRENT-LINE
     0.001 |      0.000 |             0 |           1 |   0.000993 | MAKE-PARSER
     0.001 |      0.000 |             0 |         143 |   0.000005 | LINES-OF
     0.000 |      0.000 |         4,096 |          28 |   0.000000 | INSERT-CHARACTER
     0.000 |      0.000 |             0 |          28 |   0.000000 | FLUSH-PARSERS
     0.000 |      0.000 |             0 |           1 |   0.000000 | GRAMMAR
     0.000 |      0.000 |             0 |           1 |   0.000000 | NEXT-LEX-CHARACTER
     0.000 |      0.000 |             0 |           1 |   0.000000 | NEXT-TOKEN
     0.000 |      0.000 |             0 |           1 |   0.000000 | NON-TERMINALS-IN-GRAMMAR
     0.000 |      0.000 |             0 |           1 |   0.000000 | CURRENT-LEX-CHARACTER
     0.000 |      0.000 |             0 |           1 |   0.000000 | HTML-GRAMMAR
     0.000 |      0.000 |             0 |           1 |   0.000000 | PARSE-TOKEN
     0.000 |      0.185 |   530,480,192 |  11,084,934 |   0.000000 | FIRST-OF-SYMBOL
     0.000 |      0.000 |             0 |          58 |   0.000000 | COLUMN-AT
     0.000 |      0.000 |             0 |          29 |   0.000000 | (SETF COLUMN-AT)
     0.000 |      0.000 |             0 |          30 |   0.000000 | TEXT-OF
     0.000 |      0.000 |             0 |          28 |   0.000000 | (SETF TEXT-OF)
     0.000 |      0.000 |             0 |           1 |   0.000000 | INITIAL-LR1-ITEMS-FOR-GRAMMAR
     0.000 |      0.000 |             0 |           1 |   0.000000 | ACTIONS
     0.000 |      0.943 | 2,905,081,072 |     150,542 |   0.000000 | CLOSURE-OF-LR1-ITEMS
     0.000 |      0.106 |   322,918,400 |           1 |   0.000000 | MAKE-GRAMMAR
     0.000 |      0.034 |   140,863,088 |     150,541 |   0.000000 | LR1-GOTO-FOR-SYMBOL
     0.000 |      0.000 |             0 |         117 |   0.000000 | LINE-AT
     0.000 |      0.000 |             0 |           2 |   0.000000 | MAKE-LINES
     0.000 |      0.000 |             0 |          27 |   0.000000 | (SETF PARSER-OF)
     0.000 |      0.000 |             0 |           1 |   0.000000 | INCF-LEX-POSITION
     0.000 |      0.000 |             0 |           1 |   0.000000 | PRODUCTIONS-IN-GRAMMAR
     0.000 |      0.000 |             0 |           4 |   0.000000 | STACK
     0.000 |      0.000 |             0 |           2 |   0.000000 | (SETF STACK)
     0.000 |      0.000 |         7,936 |          29 |   0.000000 | INSERT-INTO
     0.000 |      0.000 |             0 |          56 |   0.000000 | NEXT-COLUMN
     0.000 |      0.000 |         4,096 |           1 |   0.000000 | SYMBOLS-IN-GRAMMAR
     0.000 |      0.004 |     7,897,088 |           1 |   0.000000 | LR1-ITEMS-FOR-GRAMMAR
     0.000 |      0.000 |             0 |         169 |   0.000000 | CURRENT-POSITION
     0.000 |      0.000 |             0 |           1 |   0.000000 | (SETF LINES-OF)
     0.000 |      0.000 |             0 |           1 |   0.000000 | (SETF SOURCE-TEXT)
     0.000 |      0.000 |             0 |           1 |   0.000000 | INSERT-TEXT
     0.000 |      0.000 |             0 |           2 |   0.000000 | CURRENT-COLUMN-POSITION
     0.000 |      0.000 |         4,096 |           1 |   0.000000 | (SETF CURRENT-COLUMN-POSITION)
     0.000 |      0.000 |         4,096 |         172 |   0.000000 | CURRENT-LINE-POSITION
-----------------------------------------------------------------
   164.493 |      2.770 | 8,487,836,976 | 298,900,637 |            | Total

estimated total profiling overhead: 1374.35 seconds
overhead estimation parameters:
  8.000001e-9s/call, 4.598e-6s total profiling, 2.1800001e-6s internal profiling

These functions were not called:
 (SETF ACTIONS) COMPOUNDTERMP COPY-LEXER (SETF CURRENT-LINE)
 (SETF CURRENT-LINE-POSITION) (SETF CURRENT-POSITION) (SETF ENTRIES)
 EQUAL-CORES FOR-EACH-RHS FOR-EACH-RULE GET-PARSE-RESULT (SETF GOTOS) GOTOS
 (SETF GRAMMAR) GRAMMAR-419 GRAMMAR-420 INSERT-LINE (SETF LEXER) LEXER
 (SETF LINE-AT) LR1-GOTO-INDEX-FOR-SYMBOL MAKE-NEW-NAMED-RULE NEXT-LINE
 (SETF NUMBER-OF-STATES) NUMBER-OF-STATES NUMBERS-GRAMMAR PARSE-INPUT
 (SETF PARSE-POSITION) PARSE-POSITION PARSER-OF POSITIONS-EQUAL REPARSE
 (SETF SOURCE) SOURCE SOURCE-TEXT (SETF SPECIFICATION) SPECIFICATION
 SPLIT-LINE (SETF STATE) STATE (SETF STATES) STATES TERMINALS-IN-GRAMMAR
 TRANSFORM-EXTENDED-GRAMMAR-TO-FUNDAMENTAL TRANSFORM-FOR-NIL TRANSFORM-RHS
 TRANSFORM-TERM (SETF UNEXPECTED-CHARACTER) UNEXPECTED-CHARACTER
|#