-- TO DO --More testing on stray input for addRecord -- Add existance testing to the new id generator. but also make sure its easy to disable in the -- future as eventually the db should be able to repair itself and not have to worry about out of order -- GO THRUOUGH FUNCTIONS AND DECIDE ON CENVENTION FOR PARAMETER ORDER eg. needle, haystack -- MAKE PRIVATE HANDLERS PRIVATE // SETTINGS // put "FALSE" into tScrambleFileNames // put "FALSE" into tEncryptTablesOnFile // put "FALSE" into tStoreTablesInMemory private ON ____LIST_MANAGEMENT end ____LIST_MANAGEMENT function listToDBArray pList pFieldName repeat with x = 1 to the number of lines in pList put line x of pList into tContentsA[x][pFieldName] end repeat return tContentsA end listToDBArray private ON ____STORAGE_LAYER end ____STORAGE_LAYER function getTableData pTableName if (tStoreTablesInMemory) AND (pTableName is among the lines of the keys of gLoadedTablesA) then return gLoadedTablesA[pTableName] into tTableData else if tScrambleFileNames then put scrambleTableName(pTableName) into pTableName if existsTable(pTableName) then put getFile(pTableName) into tTableData else answer "TABLE DOES NOT EXIST" if tEncryptTablesOnFile then put rgDecrypt(tTableData) into tTableData put tTableData into gLoadedTablesA[pTableName] end if return tTableData end getTableData command saveTableData pTableData pTableName if tStoreTablesInMemory then put pTableData into gLoadedTablesA[pTableName] if tScrambleFileNames then put scrambleTableName(pTableName) into pTableName if tEncryptTablesOnFile then put rgEncrypt(tTableData) into tTableData writeFile tTableData, pTableName end saveTableData private ON ___CREATE end ___CREATE on createTable pTableName pStructure -- VALIDATE PSTRUCTURE, COMMA,LINE, OR "|" delimited if not(isValidStructure(pStructure)) then return "ERROR, invalid pStructure" -- SET THE DIRECTORY CORRECTLY directoryRecords -- CREATE A FILENAME if not(isValidTableName(pTableName)) then answer "Invalid TableName" exit createTable end if put pTableName into tFileName -- CHECK IF TABLE ALREAD EXISTS if existsTable(pTablename) then answer "Could Not Create Table, it already exists." exit createTable end if --NOW WE CONSTRUCT THE DATABASE STRUCTURE (THE FIRST LINE) put buildStructureString(pStructure) into tStructure -- WRITE THE FILE! writeFile tStructure,tFileName end createTable on addRecords pContentsA pTableName if not(isValidRecordsArray(pContentsA)) then answer "pContentA invalid." exit addRecords end if -- FIRST CHECK IF THE TABLE EXISTS, IF NOT CREATE IT WITH THE GIVEN if not(existsTable(pTableName)) then put the keys of pContentsA[1] into tFlds createTable pTableName, tFlds end if --Get Table Structure put getTableStructure(pTableName) into tTableStructure -- GET THE TABLE TO FIGURE OUT NEW ID AND TO APPEND NEW RECORD put getFile(pTableName) into tTable -- LOOP FOR ALL RECORDS repeat for each element tContentsA in pContentsA --ARRAY TO RECORD put arrayToRecord(tContentsA, tTableStructure) into tNewRecord --FINALLY INSERT THE ID IFF the ID is not already set. set the itemDel to "|" if item 1 of tNewRecord is empty then put theNewIDNumber(tTable) into item 1 of tNewRecord --add to table var put tNewRecord into line ((the number of lines in tTable) + 1) of tTable end repeat writeFile tTable,pTableName end addRecords on addRecord pContentsA pTableName -- DETECT IS pContentsA is a KeyValuePairList or an Array put pContentsA into tContentsA if not(isAnArray(pContentsA)) AND pContentsA contains "=" then put keyValuePairToArray(pContentsA) into tContentsA --Prepare put the keys of tContentsA into tFlds -- FIRST CHECK IF THE TABLE EXISTS, IF NOT CREATE IT WITH THE GIVEN if not(existsTable(pTableName)) then createTable pTableName, tFlds --Get Table Structure put getTableStructure(pTableName) into tTableStructure --ARRAY TO RECORD put arrayToRecord(tContentsA, tTableStructure) into tNewRecord -- GET THE TABLE TO FIGURE OUT NEW ID AND TO APPEND NEW RECORD put getFile(pTableName) into tTable --FINALLY INSERT THE ID IFF the ID is not already set. set the itemDel to "|" if item 1 of tNewRecord is empty then put theNewIDNumber(tTable) into item 1 of tNewRecord --add to table var put tNewRecord into line ((the number of lines in tTable) + 1) of tTable -- THIS IS WHERE YOU WOULD ENCRYPT -- write to disk writeFile tTable,pTableName end addRecord private ON ___READ end ___READ function getRecords pTableName -- READ FILE put getFile(pTableName) into tTable --GET THE TABLE STRUCTURE NOW BECUAE WE WILL NEED IT LATER put getTableStructureFromTable(tTable) into tTableStructure delete line 1 of tTable put 0 into tIndex repeat for each line thisRecord in tTable add 1 to tIndex put recordToArray(thisRecord, tTableStructure) into tRecordsA[tIndex] end repeat return tRecordsA end getRecords function getRecord pID pTableName -- VALIDATE if the number of lines in pID > 1 OR not(pID is a number) then answer "pID malformed" -- READ FILE put getFile(pTableName) into tTable --GET THE TABLE STRUCTURE NOW BECUAE WE WILL NEED IT LATER put getTableStructureFromTable(tTable) into tTableStructure --GET RECORD NUMBER // CREATE A FILTER TO CATCH THE RECORD ID // Building a filterPattern that matches the begining of a string with ID and pipe. note filter command doesnt use regex but filterPatternss put (pID & "|*") into tFilter -- FILTER filter tTable with tFilter --just to think out the lines of the db a bit if tTable is empty then answer "Record/s Not Found" if the number of lines in tTable > 1 then answer "Multiple Lines with same ID... OOPS!" put line 1 of tTable into tRecord end if put recordToArray(tRecord, tTableStructure) into tRecordA return tRecordA end getRecord function getFieldsIn pTableName return getTableStructure(pTableName) end getFieldsIn function getFldValues pFldName pTableName --RETURNS LIST OF ALL VALUES OF A GIVEN FLD IN A TABLE -- GET TABLE put getFile(pTableName) into tTableContents put empty into tFldValueList -- GET THE ITEM NUM put theItemNumberOfField(pFldName, pTableName) into tItemNum -- SINCE THE FIRST LINE OF THE DB IS JUST THE STRUCTURE, WE DELETE IT. delete line 1 of tTableContents -- SET UP ELABORATE LOOPING PROCEDURES THAT OPTIMIZE SPEED FOR LARGE DBS set the itemdel to "|" put the number of lines in tTableContents into tLineCount put 300 into tChunkSize put roundUp(tLineCount / tChunkSize) into tLoopTimes -- FIRST WE SPLIT UP THE tTableContents into multiple peices within an array. put 1 into x repeat tLoopTimes put line 1 to tChunkSize of tTableContents into tChunkedTable[x] delete line 1 to tChunkSize of tTableContents add 1 to x end repeat delete local tTableContents -- NOW WE RUN THROUGH EACH CHUNK OF THE TABLE, DOING WHATEVER WE HAVE TO. put 0 into x repeat tLoopTimes put x + 1 into x repeat the number of lines in tChunkedTable[x] -- DO WHATEVER CALCULATION NEEDED HERE put item tItemNum of line 1 of tChunkedTable[x] & lf after tFldValueList delete line 1 of tChunkedTable[x] end repeat end repeat return tFldValueList end getFldValues private ON ___UPDATE end ___UPDATE command updateRecord pArray pTableName -- DETECT IS pArray is a KeyValuePairList or an Array if not(isAnArray(pArray)) AND pArray contains "=" then put keyValuePairToArray(pArray) into pArray if pArray["ID"] is empty /*OR not(pArray["ID"] is an integer)*/ then answer "ERROR: Malformed Record Update Array" exit updateRecord end if put pArray["ID"] into pID put pArray into tUpdatedRecordA put getRecord(pID,pTableName) into tOldRecordA -- MERGE OLD AND NEW RECORDS AND FILTER OUT ELEMENTS IN INPUT WHICH DONT MATCH THE REORD FORMAT union tUpdatedRecordA with tOldRecordA intersect tUpdatedRecordA with tOldRecordA --DELETE THE OLD RECORD deleteRecord pID, pTableName --WRITE NEW RECORD --PROBLEM. ADD RECORD ADDS NEW RECORD ID. addRecord tUpdatedRecordA, pTableName end updateRecord command updateRecords pArray pTableName -- DETECT IS pArray is a valid update array if not(isValidRecordsArray(pArray)) then return "Error: Invalid Input" --GET THE FILE put getFile(pTableName) into tTable put getTableStructure(pTableName) into tTableStructure repeat for each element thisRecordA in pArray if thisRecordA["ID"] is empty OR not(thisRecordA["ID"] is an integer) then answer "ERROR: One of the records contains no ID or a non integer id" exit updateRecords end if put thisRecordA["ID"] into tID put thisRecordA into tUpdatedRecordA -- GET RECORD put theLineNumberOfRecord(pID, tTable) into tLineNum put recordToArray(line tLineNum of tTable, tTableStructure) into tOldRecordA -- MERGE OLD AND NEW RECORDS AND FILTER OUT ELEMENTS IN INPUT WHICH DONT MATCH THE REORD FORMAT union tUpdatedRecordA with tOldRecordA intersect tUpdatedRecordA with tOldRecordA --DELETE THE OLD RECORD put empty into line tLineNum of tTable --WRITE NEW RECORD --PROBLEM. ADD RECORD ADDS NEW RECORD ID. put arrayToRecord(tUpdatedRecordA, tTableStructure) end repeat if pArray["ID"] is empty /*OR not(pArray["ID"] is an integer)*/ then answer "ERROR: One of the records contains no ID" exit updateRecords end if put pArray["ID"] into pID put pArray into tUpdatedRecordA put getRecord(pID,pTableName) into tOldRecordA -- MERGE OLD AND NEW RECORDS AND FILTER OUT ELEMENTS IN INPUT WHICH DONT MATCH THE REORD FORMAT union tUpdatedRecordA with tOldRecordA intersect tUpdatedRecordA with tOldRecordA --DELETE THE OLD RECORD deleteRecord pID, pTableName --WRITE NEW RECORD --PROBLEM. ADD RECORD ADDS NEW RECORD ID. addRecord tUpdatedRecordA, pTableName end updateRecords private ON ___DELETE end ___DELETE command deleteRecord pID pTableName put getFile(pTableName) into tTable put theLineNumberOfRecord(pID,tTable) into tLineNum if (tLineNum is not a number) or (tLineNum < 1) then exit deleteRecord put empty into line tLineNum of tTable writeFile tTable,pTablename end deleteRecord on deleteRecord2 pID pTableName -- GET TABLE put getFile(pTableName) into tTable -- FIND AND DELETE RECORD (ASSUMES ONLY ONE OCCURENCE) -- CONSTRUCT A QUERY put pID & "|" into tQuery -- INITIAL SETTINGS FOR LOOP put FALSE into tStop put 0 into tSkipLines put 0 into tLoopCount repeat until tStop put tLoopCount + 1 into tLoopCount put lineOffset(tQuery, tTable, tSkipLines) into tLineNum put tLineNum + tSkipLines into tLineNum // The line offset function return // a relative (not absolute) line num when using a skip. this line corrects that // NOT FOUND CASE if tLineNum is 0 then answer "Error: Could not delete. No such Record Found." exit deleteRecord2 end if // FOUND, MUST CONFIRM IT IS THE ACTUAL ID if line tLineNum of tTable begins with tQuery then // CONFIRMED, DELETE LINE AND STOP LOOP delete line tLineNum of tTable put TRUE into tStop else // FALSE MATCH, START NEXT SEARCH AFTER THIS LINE put tLineNum into tSkipLines end if if tLoopCount > 50000 then put TRUE into tStop answer "Error: Runaway loop in delete records handler." end if end repeat -- WRITE OUT writeFile tTable, pTableName end deleteRecord2 command deleteTable pTableName directoryRecords //if not(pTableName is among the lines of the files) then retrun empty else return url ("file:" & pFileName) delete url ("file:" & pTableName) end deleteTable private ON _____PATHING end _____PATHING private command directoryApplication set the itemdel to "/" put the effective filename of this stack into tPath delete the last item of tPath put slash after tPath set the directory to tPath end directoryApplication private command directoryRecords directoryApplication if not("data" is among the lines of the folders) then create folder "data" set the directory to "data/" end directoryRecords private ON _____DB_UTILS end _____DB_UTILS private function existsARecordWith pValue, pFieldName, pTable return FALSE end existsARecordWith private function keyValuePairToArray pInput split pInput by return and "=" put pInput into tArray return tArray end keyValuePairToArray private function escapeChars pInput replace "|" with "[[rg-]]" in pInput replace LF with "[[rg-lf]]" in pInput replace CRLF with "[[rg-crlf]]" in pInput return pInput -- OTHER LINE FEEDS ? end escapeChars private function unescapeChars pInput -- reverse of escaped replace "[[rg-]]" with "|" in pInput replace "[[rg-lf]]" with LF in pInput replace "[[rg-crlf]]" with CRLF in pInput return pInput end unescapeChars private function theLineNumberOfRecord pID pTable put pTable into tTableCopy -- FILTER put pID & "|*" into tFilter filter tTableCopy with tFilter // NOTE THAT BECAUSE THIS IS pId|* and not *pID|* it works perfectly on just ids! -- NO MATCHES if tTableCopy is empty then return "ERROR: Record Not Found" -- MORE THAN ONE MATCH if the number of lines in tTableCopy > 1 then return "Error Multiple records with same Id" --GET LINE NUMBER put line 1 of tTablecopy into tRecord set the wholematches to true put lineOffset(tRecord, pTable) into tLineNum return tLineNum end theLineNumberOfRecord private function theRecordsMatching pField pValue pTableName end theRecordsMatching private function arrayToRecord pArray pTableStructure put the keys of pArray into tFlds put pTableStructure into tTableStructure --INIT NEW RECORD put empty into tNewRecord repeat for each line thisField in tTableStructure if thisField is among the lines of tFlds then put escapeChars(pArray[thisField]) & "|" after tNewRecord else put "|" after tNewRecord end if end repeat return tNewRecord end arrayToRecord private function recordsToArray pRecords pTableStructure filter pRecords without empty put the number of lines in pRecords into tNumRecords repeat with x = 1 to tNumRecords put recordToArray(line x of pRecords, pTableStructure) into tResultA[x] end repeat return tResultA end recordsToArray private function recordToArray pRecord pTableStructure put pTableStructure into tFlds replace "|" with return in pRecord put pRecord into tRecord -- DEPENDS ON BOTH THE FLDS AND tTable being in correct order. repeat the number of lines in tFlds put unescapeChars(line 1 of tRecord) into tRecordA[line 1 of tFlds] delete line 1 of tRecord delete line 1 of tFlds end repeat return tRecordA end recordToArray private function theItemNumberOfField pField pTableName if pField is "ID" then return 1 put getTableStructure(pTableName) into tFlds put lineOffset(pField, tFlds) into tItemNumber /*repeat with x = 2 to the number of lines in tFlds if pField is line x of tFlds then return x end repeat*/ if tItemNumber is 0 then return "ERROR: Not Found" else return tItemNumber end theItemNumberOfField private function theNewIDNumber pTableContents if pTableContents is empty then answer "Error: pFileContents is empty." -- IF ALL THAT EXISTS IS THE STRUCTURE LINE THEN WE BEGIN WITH ID 1 if the number of lines in pTableContents is 1 then return 1 -- ELSE DO THE FOLLOWING TO INCREMENT THE LAST ID set the itemdel to "|" put item 1 of the last line of pTableContents into tLastID put (tLastID + 1) into tNewID return tNewID -- FUTURE Check that tNewId does not already exist as an ID. end theNewIDNumber private function theNumberOfRecords pTableName -- GET THE FILENAME put pTableName into tFileName -- GET THE DATA put getFile(tFileName) into tFileContents if tFileContents is empty then answer pTableName & " either does not exist or is malformed." return "ERROR" end if -- SINCE THE FIRST LINE IS THE STRUCTURE, THE NUM OF RECORDS IS: put (the number of lines in tFileContents) - 1 into tNumRecords return tNumRecords end theNumberOfRecords private function buildStructureString pStructure if pStructure contains "|" and pStructure contains "," then answer "Error: pStructure contains commas and |. could not determine delimiter." exit buildStructureString end if --FIRST WE TAKE THE STRUCTURE TEXT AND PUT IT IN THE | delimited format. replace return with "|" in pStructure replace comma with "|" in pStructure replace "||" with "|" in pStructure -- INITIALIZE put empty into tStructure set the itemdel to "|" if not(item 1 of pStructure is "ID") then put "ID|" before pStructure -- NOTE THE FIRST | in |ID| is removed as the last step repeat for each item thisField in pStructure if (thisField is empty) OR (thisField contains " ") then answer "Error: A field in the Table Structure cannot be empty or contain spaces" exit buildStructureString end if if tStructure contains ("|" & thisField & "|") then answer "Error: A two fieldNames in the Table Structure cannot be the same." &lf& \ "BuildStructureString function has ignored duplicate instances of the following fieldName: " & thisField else put toUpper(thisField) & "|" after tStructure end if end repeat return tStructure end buildStructureString private function getTableStructureFromTable pTableData put pTableData into tFileContents -- GET THE STRUCTURE STRING put line 1 of tFileContents into tStructureString replace "|" with lf in tStructureString filter tStructureString without empty return tStructureString end getTableStructureFromTable private function getTableStructure pTableName -- RETURNS LINE DELIMTED LIST OF FLDS IN A TABLE -- GET THE FILENAME put pTableName into tFileName -- GET THE DATA put getFile(tFileName) into tFileContents -- GET THE STRUCTURE STRING put line 1 of tFileContents into tStructureString replace "|" with lf in tStructureString filter tStructureString without empty return tStructureString end getTableStructure ON ___DISK_OPERATIONS end ___DISK_OPERATIONS private on writeFile pFileData, pFileName directoryRecords -- GET RID OF ANY EMPTY LINES filter pFileData without empty --WRITE put pFileData into url ("file:" & pFileName) end writeFile private function getFile pFileName directoryRecords if not(pFileName is among the lines of the files) then retrun empty else return url ("file:" & pFileName) end getFile private ON ___VALIDATION end ___VALIDATION function isValidRecordsArray pContentsA if not(isAnArray(pContentsA)) then return "FALSE" if not(isAnArray(pContentsA[1])) then return "FALSE" repeat for each line thisKey in the keys of pContentsA if thisKey is not a number then return "FALSE" end repeat return "TRUE" end isValidRecordsArray private function isAnArray pInput if the keys of pInput is empty then return FALSE return TRUE end isAnArray private function isValidStructure pStructure -- CHECK FOR DUPLICATE FLD NAMES!!! return TRUE end isValidStructure function containsReservedChar pString put "/,\,?,%,*,:,|,<,>, ,.," & quote into tReserved // NOTE: THIS IMPLEMENTATION DOES NOT CURRENTLY COUNT semicolons or commas as reserved // BASED ON http://en.wikipedia.org/wiki/Filename repeat for each item thisReservedChar in tReserved if pString contains thisReservedChar then return "TRUE" end repeat return "FALSE" end containsReservedChar function isALetter pChar put charToNum(pChar) into tTemp // CHAR TO NUM AUTOMATICALLY USES FIRST CHAR OF A STRING if (tTemp >= 65 AND tTemp <= 90) OR (tTemp >= 97 AND tTemp <= 122) then return "TRUE" return "FALSE" end isALetter function isValidTableName pTablename if containsReservedChar(pTableName) then return "FALSE" return "TRUE" end isValidTableName ON ___EXISTANCE end ___EXISTANCE function existsTable pTableName -- SHOULD NOT BE A PRIVATE FUNCTION, USED IN CODE. directoryRecords if pTableName is among the lines of the files then return TRUE else return FALSE end existsTable private function existsARecordWith pValue, pFieldName, pTable return FALSE end existsARecordWith ON _____CSV_SUPPORT end _____CSV_SUPPORT function CSV2Tab pData local tNuData -- contains tabbed copy of data local tReturnPlaceholder -- replaces cr in field data to avoid line -- breaks which would be misread as records; -- replaced later during dislay local tEscapedQuotePlaceholder -- used for keeping track of quotes -- in data local tInQuotedText -- flag set while reading data between quotes -- put numtochar(11) into tReturnPlaceholder -- vertical tab as -- placeholder put numtochar(2) into tEscapedQuotePlaceholder -- used to simplify -- distinction between quotes in data and those -- used in delimiters -- -- Normalize line endings: replace crlf with cr in pData -- Win to UNIX replace numtochar(13) with cr in pData -- Mac to UNIX -- -- Put placeholder in escaped quote (non-delimiter) chars: replace ("\""e) with tEscapedQuotePlaceholder in pData replace quote"e with tEscapedQuotePlaceholder in pData -- put space before pData -- to avoid ambiguity of starting context split pData by quote put False into tInsideQuoted repeat with tCounter = 1 to the number of lines in the keys of pData put pData[tCounter] into k if (tInsideQuoted) then replace cr with tReturnPlaceholder in k put k after tNuData put False into tInsideQuoted else replace comma with tab in k put k after tNuData put true into tInsideQuoted end if end repeat -- delete char 1 of tNuData -- remove the leading space replace tEscapedQuotePlaceholder with quote in tNuData return tNuData end CSV2Tab private function convertToCSV pTableContents // cURRENTLY INCOMPLETE,SHOULD FOLLOW THIS SPEC: // http://tools.ietf.org/html/rfc4180 replace comma with "[rg-comma]" in pTableContents replace lf with crlf in pTableContents replace "|" with comma in pTableContents --GREAT WORKING FIRST STEP, CURRENT PROBLEM IS ALL COMMAS APPEAR AS [rg-Comma] return pTableContents end convertToCSV ON ___ERROR_HANDLING end ___ERROR_HANDLING function errorMessageFormat pMessage put "" before pMessage put "" after pMessage return pMessage end errorMessageFormat function encounteredError pFunctionResponse if pFunctionResponse begins with "" and pFunctionResponse ends with "" then return TRUE return FALSE end encounteredError function rgHandleError pErrorMessage pDisplayErrorTrue pStopExecutionTrue if pDisplayErrorTrue then answer pErroMessage if pStopExecutionTrue then -- ???? THis can stop a mainLoop and mess up software // send mainLoop in 3 secs // maybe something like this to resume the mainloop ? exit to top end if put rgErrorLogOutput(pErrorMessage) end rgHandleError function rgErrorLogOutput pMessage put the internet date into tTime put tTime & " : " & pMessage into tOutput end rgErrorLogOutput ON _____MATH_UTILS end _____MATH_UTILS function roundUp pNum if pNum is an integer then return pNum if not(pNum contains ".") then return "Error" set the itemdel to "." put item 1 of pNum into tInt return (tInt + 1) end roundUp function rot13 pWord put the itemdel into tDel set the itemdel to comma put "A,B,C,D,E,F,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z" into tLetters put toUpper(pWord) into pWord repeat for each char thisChar in pWord if thisChar is not among the items of tLetters then return "ERROR" // if pWord is a other charecter!@@ then return "ERROR" put charToNum(thisChar) + 13 into tNum if tNum > 90 then put (tNum - 26) into tNum put numToChar(tNum) after tResult end repeat set the itemDel to tDel return tResult end rot13