simple-sql-parser/TODO

continue 2003 review and tests

1. start replacing the 2003 stuff with 2011
2. create an error message document for the website
   - base of error messages but add some more variations
3. start thinking about tests for invalid syntax


touch up the expr hack as best as can

careful review of token parses wrt trailing delimiters/junk

undo mess in the code created by adding lots of new support:
much more documentation
refactor crufty bits
reorder the code
reconsider the names and structure of the constructors in the syntax
refactor the typename parser - it's a real mess

add documentation in Parser.lhs on the left factoring/error handling
   approach

create error message demonstration page for the website

fixes:

keyword tree, add explicit result then can use for joins also

keyword tree support prefix mode so can start from already parsed
   token

do the final big left factor: typenames, interval lits, iden +
   suffixes

left factor/try removal summary (needs updating):

identifier starts:
  interval literal
  character set literal
  typed literals, multikeywords
  identifier
  app, agg, window
  keyword function
issues in the special op internals
not between + other ops: needs new expression parsing
  not in also
  in suffix also
  lots of overlap with binary and postfix multi keyword operators
  quantified comparison also
issues in the typename parsing
dot in identifiers and as operator
issues in the symbol parser
  hardcode all the symbols in the symbol parser/split?
conflict with in suffix and in in position

rules for changing the multi keyword parsing:
  if a keyword must be followed by another
    e.g. left join, want to refactor to produce 'expected "left join"'
  if the keyword is optionally followed by another, e.g. with
   recursive, then don't do this.


rough SQL 2003 todo, including tests to write:

switch to SQL 2011

now:
review the commented out reserved keyword entries and work out how to
   fix
go through all? the functions
go through almost all the predicates
window functions missing bits, window clauses
from: more tests, review missing
  tablesample, unnest, etc.
rows review
match missing bit
between symmetric
case review

detail list from the grammar, LNR = maybe leave until after next
   release, otherwise planned for next release

LNR support needed MODULE syntax in identifiers - already covered?
LNR decide how to represent special identifiers including the session
   variables or whatever they are called like current_user
LNR grouping - needs special syntax?
LNR next value for
LNR subtypes, methods, new /routine, dereference
LNR double check associativity, precedence (value exprs, joins, set ops)
position expressions
length expressions
extract expression
cardinality expression?
check concatenations
substring expressions
regular expression substring function
convert
translate
trim
overlay
LNR specifictype
row value constructors, expressions review
review table value constructor exactly what is allowed
lots more tests for from clause variations
tablesamples
unnest
table function derived table
only spec
join variations, including union join
review group by
window clauses
LNR all fields reference with alias 'select * as (a,b,c) ... '
search or cycle clause
between symmetric/asymmetric
in predicate review
escape for like
escape for [not] similar to
LNR regular expression syntax?
normalized predicate
overlaps predicate
distinct from predicate
member predicate
submultiset predicate
set predicate
LNR type predicate
additional stuff review:
complete the list of keywords/reserved keywords and check everything
   still works ok. The parser will reject all unquoted identifiers
   which are the same as reserved or unreserved keywords.
LNR select into
LNR other language format identifiers for host params?



---

after next release

review areas where this parser is too permissive, e.g. value
   expressions allowed where column reference names only should be
   allowed, such as group by, order by (perhaps there can be a flag or
   warnings or something), unqualified asterisk in select list

fix the expression parser completely: the realistic way is to adjust
   for precedence and associativity after parsing since the concrete
   syntax is so messy. should also use this expression parser for
   parsing joins and for set operations, maybe other areas.

table expression in syntax:
  QueryExpr = Select SelectList (Maybe TableExpr)
  and the TableExpr contains all the other bits?

change the booleans in the ast to better types for less ambiguity?

decide how to handle character set literals and identifiers: don't
   have any intention of actually supporting switching character sets
   in the middle of parsing so maybe this would be better disabled?

review places in the parse which should allow only a fixed set of
   identifiers (e.g. in interval literals), keep in mind other
   dialects and extensibility

decide whether to represent numeric literals better, instead of a
   single string - break up into parts, or parse to a Decimal or
   something


= future big feature summary

all ansi sql queries
completely working expression tree parsing
error messages, left factor
dml, ddl, procedural sql
position annotation
type checker/ etc.
lexer
dialects
quasi quotes
typesafe sql dbms wrapper support for haskell
extensibility
performance analysis

try out uu-parsing or polyparse, especially wrt error message
   improvements

= stuff

try and use the proper css theme
  create a header like in the haddock with simple-sql-parser +
    contents link
  change the toc gen so that it works the same as in haddock (same
    div, no links on the actual titles
  fix the page margins, and the table stuff: patches to the css?

release checklist:
hlint
haddock review
spell check
update changelog
update website text
regenerate the examples on the index.txt

= Later general tasks:

docs

add preamble to the rendered test page

add links from the supported sql page to the rendered test page for
   each section -> have to section up the tests some more

testing

review tests to copy from hssqlppp

add lots more tests using SQL from the xb2 manual

much more table reference tests, for joins and aliases etc.?

review internal sql collection for more syntax/tests

other

----

demo program: convert tpch to sql server syntax exe processor

run through other manuals for example queries and features: sql in a
   nutshell, sql guide, sql reference guide, sql standard, sql server
   manual, oracle manual, teradata manual + re-through postgresql
   manual and make notes in each case of all syntax and which isn't
   currently supported also.

check the order of exports, imports and functions/cases in the files
fix up the import namespaces/explicit names nicely

ast checker: checks the ast represents valid syntax, the parser
   doesn't check as much as it could, and this can also be used to
   check generated trees. Maybe this doesn't belong in this package
   though?

= other sql support

top
string literals
full number literals -> other bases?
apply, pivot

other dialect targets:
postgres
oracle
teradata
ms sql server
mysql?
db2?
what other major dialects are there?
sqlite
sap dbmss (can't work out what are separate products or what are the
   dialects)