Input: Check the semantics of the <code > DDS< / code > . If <code > all< / code > is true check not only the semantics of the <code > DDS< / code > itself but also recursively check all variables in the dataset . [CODESPLIT] public void checkSemantics ( boolean all ) throws BadSemanticsException { if ( getEncodedName ( ) == null ) { DAPNode . log . error ( "A dataset must have a name" ) ; throw new BadSemanticsException ( "DDS.checkSemantics(): A dataset must have a name" ) ; } Util . uniqueNames ( vars , getEncodedName ( ) , "Dataset" ) ; if ( all ) { for ( Enumeration e = vars . elements ( ) ; e . hasMoreElements ( ) ; ) { BaseType bt = ( BaseType ) e . nextElement ( ) ; bt . checkSemantics ( true ) ; } } }


Input: Print the <code > DDS< / code > on the given <code > PrintWriter< / code > . [CODESPLIT] public void print ( PrintWriter os ) { os . println ( "Dataset {" ) ; for ( Enumeration e = vars . elements ( ) ; e . hasMoreElements ( ) ; ) { BaseType bt = ( BaseType ) e . nextElement ( ) ; bt . printDecl ( os ) ; } os . print ( "} " ) ; if ( getEncodedName ( ) != null ) os . print ( getEncodedName ( ) ) ; os . println ( ";" ) ; }


Input: Before the DDS can be used all of the Aliases in the various AttributeTables must be resolved . This means that it is necessary to verify that each Alias references an Attribute that exists and is not another Alias . This is accomplished by searching the DDS s variable s attribute holdings for Aliases Everytime an Alias is located a new search begins to find the Attribute that the Alias is attempting to reference . <p / > This method recursively searchs through the passed <code > BaseType< / code > parameter bt for Alias members of AttributeTables and when they are found attempts to resolve them to a specific Attribute . <p / > This method gets called at the top level at the parser ONLY after the entire DDS has been parsed and built . It s intial invocation get passed the DDS ( which is in fact a <code > BaseType< / code > ) <p / > <p / > This method manipulates the global variable <code > currentBT< / code > . [CODESPLIT] private void resolveAliases ( BaseType bt ) throws MalformedAliasException , UnresolvedAliasException , NoSuchAttributeException { // cache the current/parent BaseType (a container) BaseType cacheBT = currentBT ; try { // Make the one we are about to search the current one. currentBT = bt ; // Make the current AttributeTable null to indicate that we are at the top // AttributeTable of a new current BaseType. currentAT = null ; if ( Debug . isSet ( "DDS.resolveAliases" ) ) DAPNode . log . debug ( "Searching for Aliases in the Attributes of Variable: " + bt . getEncodedName ( ) ) ; // Process the Attributes of this BaseType. resolveAliases ( bt . getAttributeTable ( ) ) ; // Now if this current BaseType is a container type, then we better // search and resolve Aliases in it's children. if ( bt instanceof DConstructor ) { if ( Debug . isSet ( "DDS.resolveAliases" ) ) DAPNode . log . debug ( "Searching for Aliases in the children of Variable: " + bt . getEncodedName ( ) ) ; Enumeration bte = ( ( DConstructor ) bt ) . getVariables ( ) ; while ( bte . hasMoreElements ( ) ) { BaseType thisBT = ( BaseType ) bte . nextElement ( ) ; // Recursive call... resolveAliases ( thisBT ) ; } } } finally { // Restore the previous current BaseType state. currentBT = cacheBT ; } }


Input: This method recursively searchs through the passed <code > AttributeTable< / code > parameter at for Alias members . When an Alias is found the method attempts to resolve it to a specific Attribute . <p / > This method is invoked by <code > resolveAliases ( BaseType bt ) < / code > and is used to search for Aliases in AttributeTables found in a BaseTypes Attributes . <p / > This method manipulates the global variable <code > currentBT< / code > . [CODESPLIT] private void resolveAliases ( AttributeTable at ) throws MalformedAliasException , UnresolvedAliasException , NoSuchAttributeException { // Cache the current (parent) Attribute table. This value is // null if this method is call from resolveAliases(BasetType bt) AttributeTable cacheAT = currentAT ; try { // Set the current AttributeTable to the one that we are searching. currentAT = at ; //getall of the Attributes from the table. Enumeration aNames = currentAT . getNames ( ) ; while ( aNames . hasMoreElements ( ) ) { String aName = ( String ) aNames . nextElement ( ) ; opendap . dap . Attribute thisA = currentAT . getAttribute ( aName ) ; if ( thisA . isAlias ( ) ) { //Is Alias? Resolve it! resolveAlias ( ( Alias ) thisA ) ; if ( Debug . isSet ( "DDS.resolveAliases" ) ) { DAPNode . log . debug ( "Resolved Alias: '" + thisA . getEncodedName ( ) + "'\n" ) ; } } else if ( thisA . isContainer ( ) ) { //Is AttributeTable (container)? Search it! resolveAliases ( thisA . getContainer ( ) ) ; } } } finally { // Restore the previous currentAT state. currentAT = cacheAT ; } }


Input: This method attempts to resolve the past Alias to a specific Attribute in the DDS . It does this by : <ul > <li > 1 ) Tokenizing the Alias s attribute field ( see <code > Alias< / code > ) < / li > <li > 2 ) Evaluating the tokenized field to locate the longest possible variable name represented as a consecutive set of tokens < / li > <li > 2 ) Evaluating the the remaining tokenized field to locate the Attribute that this Alias is attempting to reference< / li > <li > 4 ) Setting the Aliases internal references for it s Variable and it s Attribute . < / ul > <p / > If an Attribute matching the definition of the Alias cannot be located an Exception is thrown [CODESPLIT] private void resolveAlias ( Alias alias ) throws MalformedAliasException , UnresolvedAliasException { //Get the crucial stuff out of the Alias String name = alias . getEncodedName ( ) ; String attribute = alias . getAliasedToAttributeFieldAsClearString ( ) ; if ( Debug . isSet ( "DDS.resolveAliases" ) ) { DAPNode . log . debug ( "\n\nFound: Alias " + name + "  " + attribute ) ; } // The Attribute field MAY NOT be empty. if ( attribute . equals ( "" ) ) { throw new MalformedAliasException ( "The attribute 'attribute' in the Alias " + "element (name: '" + name + "') must have a value other than an empty string." ) ; } if ( Debug . isSet ( "DDS.resolveAliases" ) ) { DAPNode . log . debug ( "Attribute: `" + attribute + "'" ) ; } // Tokenize the attribute field. Vector aNames = tokenizeAliasField ( attribute ) ; if ( Debug . isSet ( "DDS.resolveAliases" ) ) { DAPNode . log . debug ( "Attribute name tokenized to " + aNames . size ( ) + " elements" ) ; Enumeration e = aNames . elements ( ) ; while ( e . hasMoreElements ( ) ) { String aname = ( String ) e . nextElement ( ) ; DAPNode . log . debug ( "name: " + aname ) ; } } // The variable reference is the first part of the attribute field. // Let's go find it... BaseType targetBT = null ; // Absolute paths for attributes names must start with the dot character. boolean isAbsolutePath = aNames . get ( 0 ) . equals ( "." ) ; if ( ! isAbsolutePath ) { //Is it not an absolute path? throw new MalformedAliasException ( "In the Alias '" + name + "'" + " the value of the attribute 'attribute' does not begin with the character dot (.). " + "The value of the 'attribute' field must always be an absolute path name from the " + "top level of the variable reference, and thus must always begin with the dot (.) character." ) ; } if ( aNames . size ( ) == 1 ) { // Is it only a dot? throw new MalformedAliasException ( "In the Alias '" + name + "'" + " the value of the attribute 'attribute' contains only the character dot (.). " + "The value of the 'attribute' field must always reference an Attribute using an absolute path name from the " + "top level of the DAS, and must reference an attribute within the DAS. A simple dot is not allowed." ) ; } aNames . remove ( 0 ) ; // Remove the first token, which by now we know is a single dot. targetBT = getDeepestMatchingVariable ( this , aNames ) ; if ( targetBT == null ) { // No matching BaseType? // Then assume the attribute field references a // top (Dataset) level Attribute. targetBT = this ; } //LogStream.out.println("Alias references variable:	."+targetBT.getLongName()); // Now that we have found a target BaseType variable that matches the reference in // the variable field of the Alias (b.t.w. that's a good thing) let's go // see if we can find an Attribute within that targetBT that matches the attribute field // in the Alias decleration. Attribute targetAT = null ; if ( aNames . size ( ) == 0 ) { // If there are no remaining tokens in the attribute field then // we are referencing the attribute container of the targetBT. targetAT = targetBT . getAttribute ( ) ; } else { // Go try to find the Attribute in the targetBT. targetAT = getAttribute ( targetBT . getAttributeTable ( ) , aNames ) ; } alias . setMyVariable ( targetBT ) ; alias . setMyAttribute ( targetAT ) ; }


Input: This method executes a ( recursive ) search of the <code > DConstructor< / code > parameter <b > dcBT< / b > for a <code > BaseType< / code > variable whose name resolves to the vector of names contained in the <code > Vector< / code > parameter <b > vNames< / b > . A variable is considered a match if each of it s node names in the hierarchy of containers in the one passed as parameter <b > dcBT< / b > matches ( equals ) the corresponding name in the Vector <b > vNames< / b > . [CODESPLIT] private BaseType getDeepestMatchingVariable ( DConstructor dcBT , Vector vNames ) { // Get the first name from the Vector String vName = ( String ) vNames . get ( 0 ) ; // Get all of the child variables from the Dconstructor Enumeration bte = dcBT . getVariables ( ) ; while ( bte . hasMoreElements ( ) ) { // Get this variable BaseType bt = ( BaseType ) bte . nextElement ( ) ; // Get and normalize it's name. String normName = normalize ( bt . getClearName ( ) ) ; // Compare the names if ( normName . equals ( vName ) ) { // They match! // Remove the name from the vector. vNames . remove ( 0 ) ; if ( vNames . size ( ) == 0 ) { // are there more names? // Nope! We Found it! return bt ; } if ( bt instanceof DConstructor ) { // If there are more names then this thing better be a container // recursively search it for the remaining names... BaseType nextBT = getDeepestMatchingVariable ( ( DConstructor ) bt , vNames ) ; if ( nextBT != null ) return ( nextBT ) ; return ( bt ) ; } return ( bt ) ; } } return ( null ) ; }


Input: The <code > normalize< / code > method is used to normalize variable and attribute name strings prior to their comparison with the normalized tokens extracted from the variable and name fields in an Alias declaration . <p / > The rule for this normalization is as follows : <p / > <ul > <li > The &quot ; ( double quote ) and the \ ( backslash aka escape ) characters MUST be escaped ( using the \ character ) in the <b > variable< / b > and <b > attribute< / b > fields . < / li > < / ul > [CODESPLIT] public static String normalize ( String field ) { boolean Debug = false ; StringBuffer sb = new StringBuffer ( field ) ; for ( int offset = 0 ; offset < sb . length ( ) ; offset ++ ) { char c = sb . charAt ( offset ) ; // for every quote and slach in the string, add a slash in front of it. if ( c == slash || c == quote ) { sb . insert ( offset , slash ) ; offset ++ ; } } //Coverity[DEADCODE] if ( Debug ) { DAPNode . log . debug ( "String: `" + field + "` normalized to: `" + sb + "`" ) ; } return ( sb . toString ( ) ) ; }


Input: The <code > tokenizeAliasFiled () < / code > method is used to tokenize the <b > variable< / b > and the <b > attribute< / b > fields in the alias declaration . It is required that these fields be <b > normalized< / b > in the XML instance document . The rules for this normalization are as follows : <ul > <p / > <li > The &quot ; ( double quote ) and the \ ( backslash aka escape ) characters MUST be escaped ( using the \ character ) in the <b > variable< / b > and <b > attribute< / b > fields . < / li > <p / > <li > The <b > variable< / b > and <b > attribute< / b > fields must be enclosed in double quotes if their values contain the dot ( . ) character . < / li > <p / > <li > Fully qualified <b > variable< / b > and <b > attribute< / b > names always begin with the dot ( . ) character . < / li > < / ul > [CODESPLIT] public static Vector tokenizeAliasField ( String field ) throws MalformedAliasException { boolean Debug = false ; // find the index of the last element in the field. int lastIndex = field . length ( ) - 1 ; // make a place to put the tokens. Vector tokens = new Vector ( ) ; //Coverity[DEADCODE] if ( Debug ) { DAPNode . log . debug ( "lastIndexOf(dot): " + field . lastIndexOf ( dot ) + "   lastIndex: " + lastIndex ) ; } // Does this thing start with a quote? if ( field . charAt ( 0 ) == quote ) { // find the closing quote. // Because this token starts with a quote, it must be normalized // (see method description). The closing quote must exist, // and it cannont be escaped. // The first character in the token is the one following the // leadin quote. int start = 1 ; // prepare to search for a closing quote. int end = - 1 ; boolean done = false ; boolean escaped = false ; // search for the quote for ( int i = 1 ; i <= lastIndex || ! done ; i ++ ) { char c = field . charAt ( i ) ; //LogStream.out.println("Checking for clear quote on char: "+c+" escaped="+escaped+"  done="+done); // Was this character escaped (with a slash)? if ( escaped ) { // then ignore it and unset the escaped flag // since the escape has been consumed. escaped = false ; } else { // otherwise, is it an escape (slash) character if ( c == slash ) { // the set the escaoed flag to true. escaped = true ; } else if ( c == quote ) { // if it's not an escape (slash) then is it a quote? //LogStream.out.println("Found quote!"); end = i ; done = true ; } } } //LogStream.out.println("start="+start+"  end="+end+"  lastIndex="+lastIndex); // if the end is less than 0 then it didn't get set // during the search for the quote, and thus the closing quote wasn't // found. Throw an exception! if ( end < 0 ) throw new MalformedAliasException ( "Alias fields that begin with the quote (\") sign " + "must have a closing quote." ) ; // If there is more stuff, and that stuff is not seperated from the // closing quote by a dot character, then it's bad syntax. if ( lastIndex > end && field . charAt ( end + 1 ) != dot ) throw new MalformedAliasException ( "Alias fields must be seperated by the dot (.) character." ) ; // The last caharcter in the field may not be an (unquoted) dot. if ( field . charAt ( lastIndex ) == dot ) throw new MalformedAliasException ( "Alias fields may not end with the dot (.) character." ) ; // Looks like we found a complete token. // Get it. String firstToken = field . substring ( start , end ) ; // Add it to the tokens Vector. tokens . add ( firstToken ) ; // if there is more stuff, then tokenize it. if ( end < lastIndex ) { // get the rest of the stuff String theRest = field . substring ( end + 2 ) ; // tokenize it and add each of the returned tokens to // this tokens Vector. // Recursive call. Enumeration tkns = tokenizeAliasField ( theRest ) . elements ( ) ; while ( tkns . hasMoreElements ( ) ) tokens . add ( tkns . nextElement ( ) ) ; } return ( tokens ) ; } // Find the first dot. This simplistic search is appropriate because // if this field contained a dot as part of it's name it should have // been encased in quotes and handled by the previous logic. int firstDot = field . indexOf ( dot ) ; if ( firstDot == 0 ) { // Does this thing start with dot? // Then it must be an absolute path. // NOTE: This should be true ONLY for the first token // in the list. By that I mean that a leading dot in // the field string should only occur when the // variable or alias field begins a dot. A secondary // token may only start with a dot if the dot is // actually part of the field, and thus it should be // encased in quotes. String thisToken = "." ; tokens . add ( thisToken ) ; // Check to see if there are more characters in the field to be tokenized. // If there are, tokenize them. if ( lastIndex > 0 ) { String theRest = field . substring ( 1 ) ; // Recursive call Enumeration tkns = tokenizeAliasField ( theRest ) . elements ( ) ; // Take the tokens from the rest of the fields and // add them to this token vector. while ( tkns . hasMoreElements ( ) ) tokens . add ( tkns . nextElement ( ) ) ; } return ( tokens ) ; } if ( firstDot > 0 ) { // A secondary token may only contain a dot if the dot is // actually part of the field, and thus the field should have been // encased in quotes. Since we already check for a leading quote, // the first dor MUST be the end of the token. String firstToken = field . substring ( 0 , firstDot ) ; tokens . add ( firstToken ) ; // A quick syntax check. if ( lastIndex == firstDot ) throw new MalformedAliasException ( "Alias fields may not end with the dot (.) character." ) ; // Get the rest of the field string String theRest = field . substring ( firstDot + 1 ) ; // tokenize it, and add it's tokens to this token Vector. Enumeration tkns = tokenizeAliasField ( theRest ) . elements ( ) ; while ( tkns . hasMoreElements ( ) ) tokens . add ( tkns . nextElement ( ) ) ; return ( tokens ) ; } // This field string might be the final token, if we // get here it must be so add it to the tokens vector tokens . add ( field ) ; return ( tokens ) ; }


Input: Prints the peristent representation of the <code > DDS< / code > as an XML document . This XML document is know as a <b > DDX< / b > . The DDX can be parsed using the <code > DDSXMLParser< / code > [CODESPLIT] public void printXML ( PrintWriter pw , String pad , boolean constrained ) { pw . println ( "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" ) ; pw . print ( "<Dataset" ) ; if ( getEncodedName ( ) != null ) pw . print ( " name=\"" + DDSXMLParser . normalizeToXML ( getEncodedName ( ) ) + "\"" ) ; pw . println ( ) ; pw . println ( "xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"" ) ; pw . println ( "xmlns=\"" + opendapNameSpace + "\"" ) ; pw . print ( "xsi:schemaLocation=\"" ) ; pw . print ( opendapNameSpace + "  " ) ; pw . print ( schemaLocation ) ; pw . println ( "\" >" ) ; pw . println ( "" ) ; Enumeration e = getAttributeNames ( ) ; while ( e . hasMoreElements ( ) ) { String aName = ( String ) e . nextElement ( ) ; Attribute a = getAttribute ( aName ) ; if ( a != null ) a . printXML ( pw , pad + "\t" , constrained ) ; } pw . println ( "" ) ; Enumeration ve = getVariables ( ) ; while ( ve . hasMoreElements ( ) ) { BaseType bt = ( BaseType ) ve . nextElement ( ) ; bt . printXML ( pw , pad + "\t" , constrained ) ; } pw . println ( "" ) ; if ( _dataBlobID != null ) { pw . println ( pad + "\t" + "<dataBLOB href=\"" + DDSXMLParser . normalizeToXML ( _dataBlobID ) + "\"/>" ) ; } pw . println ( pad + "</Dataset>" ) ; }


Input: Takes the passed parameter <code > das< / code > and attempts to incorporate it s contents into the Attributes of the DDS variables . If an <code > Attribute< / code > in the <code > DAS< / code > can t be associated with a variable in a logical manner then it is placed at the top level of the DDS . ( Basically it becomes a toplevel attribute in the dataset ) [CODESPLIT] public void ingestDAS ( DAS das ) { try { ingestAttributeTable ( das , this ) ; resolveAliases ( ) ; } catch ( DASException de ) { DAPNode . log . error ( "DDS.ingestDAS(): " + de . getMessage ( ) ) ; } }


Input: A helper methods for ingestDAS () . [CODESPLIT] private void ingestAttribute ( Attribute a , BaseType bt ) throws DASException { if ( a . isAlias ( ) ) { // copy an alias. String name = a . getEncodedName ( ) ; String attribute = ( ( Alias ) a ) . getAliasedToAttributeFieldAsClearString ( ) ; bt . addAttributeAlias ( name , attribute ) ; } else if ( a . isContainer ( ) ) { AttributeTable at = a . getContainer ( ) ; ingestAttributeTable ( at , bt ) ; } else { // copy an Attribute and it's values... String name = a . getEncodedName ( ) ; int type = a . getType ( ) ; Enumeration vals = a . getValues ( ) ; while ( vals . hasMoreElements ( ) ) { String value = ( String ) vals . nextElement ( ) ; bt . appendAttribute ( name , type , value , true ) ; } } }


Input: A helper methods for ingestDAS () . [CODESPLIT] private void ingestAttributeTable ( AttributeTable at , DConstructor dc ) throws DASException { Enumeration ate = at . getNames ( ) ; while ( ate . hasMoreElements ( ) ) { String aName = ( String ) ate . nextElement ( ) ; Attribute a = at . getAttribute ( aName ) ; boolean foundIt = false ; Enumeration bte = dc . getVariables ( ) ; while ( bte . hasMoreElements ( ) ) { BaseType thisBT = ( BaseType ) bte . nextElement ( ) ; String bName = thisBT . getEncodedName ( ) ; if ( bName . equals ( aName ) ) { if ( a . isContainer ( ) && thisBT instanceof DConstructor ) { ingestAttributeTable ( a . getContainer ( ) , ( DConstructor ) thisBT ) ; } else { ingestAttribute ( a , thisBT ) ; } foundIt = true ; } } if ( ! foundIt ) { ingestAttribute ( a , dc ) ; } } }


Input: A helper methods for ingestDAS () . [CODESPLIT] private void ingestAttributeTable ( AttributeTable at , BaseType bt ) throws DASException { try { String atName = at . getEncodedName ( ) ; String bName = bt . getEncodedName ( ) ; //LogStream.out.println("ingestATTbl: atName:"+atName+" bName: "+bName); if ( bName . equals ( atName ) ) { //LogStream.out.println("adding each attribute!"); Enumeration e = at . getNames ( ) ; while ( e . hasMoreElements ( ) ) { String aName = ( String ) e . nextElement ( ) ; Attribute a = at . getAttribute ( aName ) ; ingestAttribute ( a , bt ) ; } } else { //LogStream.out.println("addingcontainer!"); bt . addAttributeContainer ( at ) ; } } catch ( AttributeExistsException ase ) { Enumeration e = at . getNames ( ) ; while ( e . hasMoreElements ( ) ) { String aName = ( String ) e . nextElement ( ) ; Attribute a = at . getAttribute ( aName ) ; ingestAttribute ( a , bt ) ; } } }


Input: Check for name conflicts . In the XML representation of the DDS it is syntactically possible for a variable container ( Dconstructor ) to possess an Attribute that has the same name as one of the container variable s member variables . That s a NO - NO! . Check for it here and throw a nice fat exception if we find it . [CODESPLIT] private void checkForAttributeNameConflict ( DConstructor dc ) throws BadSemanticsException { if ( _Debug ) { DAPNode . log . debug ( "Checking " + dc . getTypeName ( ) + " " + dc . getClearName ( ) + " for name conflicts." ) ; } Enumeration bte = dc . getVariables ( ) ; while ( bte . hasMoreElements ( ) ) { BaseType bt = ( BaseType ) bte . nextElement ( ) ; //LogStream.out.println("     member: "+bt.getTypeName()+" "+bt.getName()); Enumeration ate = dc . getAttributeNames ( ) ; while ( ate . hasMoreElements ( ) ) { String aName = ( String ) ate . nextElement ( ) ; //LogStream.out.println("         attribute: "+aName); if ( aName . equals ( bt . getEncodedName ( ) ) ) { throw new BadSemanticsException ( "The variable '" + dc . getLongName ( ) + "' has an Attribute with the same name ('" + aName + "') as one of it's " + "member variables (" + bt . getTypeName ( ) + " " + bt . getEncodedName ( ) + ")\n" + "This is NOT allowed." ) ; } } if ( bt instanceof DConstructor ) { // LogStream.out.println("     member '"+bt.getName()+"' is a container. Better Check it!"); // Recursive call!! checkForAttributeNameConflict ( ( DConstructor ) bt ) ; } } }


Input: This a wrapper method for <code > DDS . print () < / code > . [CODESPLIT] public String getDDSText ( ) { StringWriter sw = new StringWriter ( ) ; this . print ( new PrintWriter ( sw ) ) ; return sw . toString ( ) ; }


Input: This a wrapper method for <code > DDS . printXML () < / code > . [CODESPLIT] public String getDDXText ( ) { StringWriter sw = new StringWriter ( ) ; this . printXML ( new PrintWriter ( sw ) ) ; return sw . toString ( ) ; }


Input: Returns a clone of this <code > DDS< / code > . See DAPNode . cloneDag () [CODESPLIT] public DAPNode cloneDAG ( CloneMap map ) throws CloneNotSupportedException { DDS d = ( DDS ) super . cloneDAG ( map ) ; d . vars = new Vector ( ) ; for ( int i = 0 ; i < vars . size ( ) ; i ++ ) { BaseType element = ( BaseType ) vars . elementAt ( i ) ; d . vars . addElement ( cloneDAG ( map , element ) ) ; } d . setEncodedName ( this . getEncodedName ( ) ) ; // Question: // What about copying the BaseTypeFactory? // Do we want a reference to the same one? Or another             // Is there a difference? Should we be building the clone // using "new DDS(getFactory())"?? // Answer: // Yes. Use the same type factory! d . factory = this . factory ; return d ; }


Input:
LOOK what about extending an index ?? [CODESPLIT] public boolean makeIndex ( String filename , RandomAccessFile dataRaf ) throws IOException { String idxPath = filename ; if ( ! idxPath . endsWith ( GBX9_IDX ) ) idxPath += GBX9_IDX ; File idxFile = GribIndexCache . getFileOrCache ( idxPath ) ; File idxFileTmp = GribIndexCache . getFileOrCache ( idxPath + ".tmp" ) ; boolean ok = false ; RandomAccessFile raf = null ; try ( FileOutputStream fout = new FileOutputStream ( idxFileTmp ) ) { //// header message
 fout . write ( MAGIC_START . getBytes ( CDM . utf8Charset ) ) ; NcStream . writeVInt ( fout , version ) ; Map < Long , Integer > gdsMap = new HashMap <> ( ) ; gdsList = new ArrayList <> ( ) ; records = new ArrayList <> ( 200 ) ; Grib2IndexProto . Grib2Index . Builder rootBuilder = Grib2IndexProto . Grib2Index . newBuilder ( ) ; rootBuilder . setFilename ( filename ) ; if ( dataRaf == null ) { raf = RandomAccessFile . acquire ( filename ) ; dataRaf = raf ; } Grib2RecordScanner scan = new Grib2RecordScanner ( dataRaf ) ; while ( scan . hasNext ( ) ) { Grib2Record r = scan . next ( ) ; if ( r == null ) break ; // done
 records . add ( r ) ; Grib2SectionGridDefinition gdss = r . getGDSsection ( ) ; Integer index = gdsMap . get ( gdss . calcCRC ( ) ) ; if ( index == null ) { gdsList . add ( gdss ) ; index = gdsList . size ( ) - 1 ; gdsMap . put ( gdss . calcCRC ( ) , index ) ; rootBuilder . addGdsList ( makeGdsProto ( gdss ) ) ; } rootBuilder . addRecords ( makeRecordProto ( r , index , r . getGDS ( ) . scanMode ) ) ; } if ( records . isEmpty ( ) ) throw new RuntimeException ( "No GRIB2 records found in " + dataRaf . getLocation ( ) ) ; Grib2IndexProto . Grib2Index index = rootBuilder . build ( ) ; byte [ ] b = index . toByteArray ( ) ; NcStream . writeVInt ( fout , b . length ) ; // message size
 fout . write ( b ) ; // message  - all in one gulp
 logger . debug ( "  made gbx9 index for {} size={}" , filename , b . length ) ; ok = true ; return true ; } finally { if ( raf != null ) raf . close ( ) ; // only close if we opened it
 // now switch; fout has been closed
 if ( ok ) { RandomAccessFile . eject ( idxFile . getPath ( ) ) ; boolean deleteOk = ! idxFile . exists ( ) || idxFile . delete ( ) ; boolean renameOk = idxFileTmp . renameTo ( idxFile ) ; if ( ! deleteOk ) logger . error ( "  could not delete Grib2Index= {}" , idxFile . getPath ( ) ) ; if ( ! renameOk ) logger . error ( "  could not rename Grib2Index= {}" , idxFile . getPath ( ) ) ; } } }


Input: / * message GribIdSection { required uint32 center_id = 1 ; required uint32 subcenter_id = 2 ; required uint32 master_table_version = 3 ; required uint32 local_table_version = 4 ; required uint32 significanceOfRT = 5 ; repeated uint32 refDate = 6 [ packed = true ] ; // year month day hour minute second ; required uint32 productionStatus = 7 ; required uint32 processedDataType = 8 ; } [CODESPLIT] private Grib2IndexProto . GribIdSection makeIdProto ( Grib2SectionIdentification id ) { Grib2IndexProto . GribIdSection . Builder b = Grib2IndexProto . GribIdSection . newBuilder ( ) ; b . setCenterId ( id . getCenter_id ( ) ) ; b . setSubcenterId ( id . getSubcenter_id ( ) ) ; b . setMasterTableVersion ( id . getMaster_table_version ( ) ) ; b . setLocalTableVersion ( id . getLocal_table_version ( ) ) ; b . setSignificanceOfRT ( id . getSignificanceOfRT ( ) ) ; b . addRefDate ( id . getYear ( ) ) ; b . addRefDate ( id . getMonth ( ) ) ; b . addRefDate ( id . getDay ( ) ) ; b . addRefDate ( id . getHour ( ) ) ; b . addRefDate ( id . getMinute ( ) ) ; b . addRefDate ( id . getSecond ( ) ) ; b . setProductionStatus ( id . getProductionStatus ( ) ) ; b . setProcessedDataType ( id . getTypeOfProcessedData ( ) ) ; return b . build ( ) ; }


Input: Write GridDatatype data to the geotiff file . [CODESPLIT] public void writeGrid ( GridDataset dataset , GridDatatype grid , Array data , boolean greyScale ) throws IOException { GridCoordSystem gcs = grid . getCoordinateSystem ( ) ; if ( ! gcs . isRegularSpatial ( ) ) { throw new IllegalArgumentException ( "Must have 1D x and y axes for " + grid . getFullName ( ) ) ; } CoordinateAxis1D xaxis = ( CoordinateAxis1D ) gcs . getXHorizAxis ( ) ; CoordinateAxis1D yaxis = ( CoordinateAxis1D ) gcs . getYHorizAxis ( ) ; // units may need to be scaled to meters double scaler = ( xaxis . getUnitsString ( ) . equalsIgnoreCase ( "km" ) ) ? 1000.0 : 1.0 ; // data must go from top to bottom double xStart = xaxis . getCoordEdge ( 0 ) * scaler ; double yStart = yaxis . getCoordEdge ( 0 ) * scaler ; double xInc = xaxis . getIncrement ( ) * scaler ; double yInc = Math . abs ( yaxis . getIncrement ( ) ) * scaler ; if ( yaxis . getCoordValue ( 0 ) < yaxis . getCoordValue ( 1 ) ) { data = data . flip ( 0 ) ; yStart = yaxis . getCoordEdge ( ( int ) yaxis . getSize ( ) ) * scaler ; } if ( ! xaxis . isRegular ( ) || ! yaxis . isRegular ( ) ) { throw new IllegalArgumentException ( "Must be evenly spaced grid = " + grid . getFullName ( ) ) ; } if ( pageNumber > 1 ) { geotiff . initTags ( ) ; } // write it out writeGrid ( grid , data , greyScale , xStart , yStart , xInc , yInc , pageNumber ) ; pageNumber ++ ; }


Input: Write Grid data to the geotiff file . Grid currently must : <ol > <li > have a 1D X and Y coordinate axes . <li > be lat / lon or Lambert Conformal Projection <li > be equally spaced < / ol > [CODESPLIT] protected void writeGrid ( GridDatatype grid , Array data , boolean greyScale , double xStart , double yStart , double xInc , double yInc , int imageNumber ) throws IOException { int nextStart = 0 ; GridCoordSystem gcs = grid . getCoordinateSystem ( ) ; // get rid of this when all projections are implemented if ( ! gcs . isLatLon ( ) && ! ( gcs . getProjection ( ) instanceof LambertConformal ) && ! ( gcs . getProjection ( ) instanceof Stereographic ) && ! ( gcs . getProjection ( ) instanceof Mercator ) //  && !(gcs.getProjection() instanceof TransverseMercator)   LOOK broken ?? && ! ( gcs . getProjection ( ) instanceof AlbersEqualAreaEllipse ) && ! ( gcs . getProjection ( ) instanceof AlbersEqualArea ) ) { throw new IllegalArgumentException ( "Unsupported projection = " + gcs . getProjection ( ) . getClass ( ) . getName ( ) ) ; } // write the data first MAMath . MinMax dataMinMax = grid . getMinMaxSkipMissingData ( data ) ; if ( greyScale ) { ArrayByte result = replaceMissingValuesAndScale ( grid , data , dataMinMax ) ; nextStart = geotiff . writeData ( ( byte [ ] ) result . getStorage ( ) , imageNumber ) ; } else { ArrayFloat result = replaceMissingValues ( grid , data , dataMinMax ) ; nextStart = geotiff . writeData ( ( float [ ] ) result . getStorage ( ) , imageNumber ) ; } // set the width and the height int height = data . getShape ( ) [ 0 ] ; // Y int width = data . getShape ( ) [ 1 ] ; // X writeMetadata ( greyScale , xStart , yStart , xInc , yInc , height , width , imageNumber , nextStart , dataMinMax , gcs . getProjection ( ) ) ; }


Input: Replace missing values with dataMinMax . min - 1 . 0 ; return a floating point data array . [CODESPLIT] private ArrayFloat replaceMissingValues ( IsMissingEvaluator grid , Array data , MAMath . MinMax dataMinMax ) { float minValue = ( float ) ( dataMinMax . min - 1.0 ) ; ArrayFloat floatArray = ( ArrayFloat ) Array . factory ( DataType . FLOAT , data . getShape ( ) ) ; IndexIterator dataIter = data . getIndexIterator ( ) ; IndexIterator floatIter = floatArray . getIndexIterator ( ) ; while ( dataIter . hasNext ( ) ) { float v = dataIter . getFloatNext ( ) ; if ( grid . isMissing ( ( double ) v ) ) { v = minValue ; } floatIter . setFloatNext ( v ) ; } return floatArray ; }


Input: Replace missing values with 0 ; scale other values between 1 and 255 return a byte data array . [CODESPLIT] private ArrayByte replaceMissingValuesAndScale ( IsMissingEvaluator grid , Array data , MAMath . MinMax dataMinMax ) { double scale = 254.0 / ( dataMinMax . max - dataMinMax . min ) ; ArrayByte byteArray = ( ArrayByte ) Array . factory ( DataType . BYTE , data . getShape ( ) ) ; IndexIterator dataIter = data . getIndexIterator ( ) ; IndexIterator resultIter = byteArray . getIndexIterator ( ) ; byte bv ; while ( dataIter . hasNext ( ) ) { double v = dataIter . getDoubleNext ( ) ; if ( grid . isMissing ( v ) ) { bv = 0 ; } else { int iv = ( int ) ( ( v - dataMinMax . min ) * scale + 1 ) ; bv = ( byte ) ( iv & 0xff ) ; } resultIter . setByteNext ( bv ) ; } return byteArray ; }


Input: LOOK WTF ?? is this the seam crossing ?? [CODESPLIT] private double geoShiftGetXstart ( Array lon , double inc ) { Index ilon = lon . getIndex ( ) ; int [ ] lonShape = lon . getShape ( ) ; IndexIterator lonIter = lon . getIndexIterator ( ) ; double xlon = 0.0 ; LatLonPoint p0 = new LatLonPointImpl ( 0 , lon . getFloat ( ilon . set ( 0 ) ) ) ; LatLonPoint pN = new LatLonPointImpl ( 0 , lon . getFloat ( ilon . set ( lonShape [ 0 ] - 1 ) ) ) ; xlon = p0 . getLongitude ( ) ; while ( lonIter . hasNext ( ) ) { float l = lonIter . getFloatNext ( ) ; LatLonPoint pn = new LatLonPointImpl ( 0 , l ) ; if ( pn . getLongitude ( ) < xlon ) { xlon = pn . getLongitude ( ) ; } } if ( p0 . getLongitude ( ) == pN . getLongitude ( ) ) { xlon = xlon - inc ; } return xlon ; }


Input:
Write GridCoverage data to the geotiff file . [CODESPLIT] public void writeGrid ( GeoReferencedArray array , boolean greyScale ) throws IOException { CoverageCoordSys gcs = array . getCoordSysForData ( ) ; if ( ! gcs . isRegularSpatial ( ) ) throw new IllegalArgumentException ( "Must have 1D x and y axes for " + array . getCoverageName ( ) ) ; Projection proj = gcs . getProjection ( ) ; CoverageCoordAxis1D xaxis = ( CoverageCoordAxis1D ) gcs . getXAxis ( ) ; CoverageCoordAxis1D yaxis = ( CoverageCoordAxis1D ) gcs . getYAxis ( ) ; // latlon coord does not need to be scaled double scaler = ( xaxis . getUnits ( ) . equalsIgnoreCase ( "km" ) ) ? 1000.0 : 1.0 ; // data must go from top to bottom double xStart = xaxis . getCoordEdge1 ( 0 ) * scaler ; double yStart = yaxis . getCoordEdge1 ( 0 ) * scaler ; double xInc = xaxis . getResolution ( ) * scaler ; double yInc = Math . abs ( yaxis . getResolution ( ) ) * scaler ; Array data = array . getData ( ) . reduce ( ) ; if ( yaxis . getCoordMidpoint ( 0 ) < yaxis . getCoordMidpoint ( 1 ) ) { data = data . flip ( 0 ) ; yStart = yaxis . getCoordEdgeLast ( ) ; } /*  remove - i think unneeded, monotonic lon handled in CoordinateAxis1D. JC 3/18/2013
     if (gcs.isLatLon()) {
      Array lon = xaxis.read();
      data = geoShiftDataAtLon(data, lon);
      xStart = geoShiftGetXstart(lon, xInc);
      //xStart = -180.0;
    }  */ if ( pageNumber > 1 ) { geotiff . initTags ( ) ; } // write the data first int nextStart = 0 ; MAMath . MinMax dataMinMax = MAMath . getMinMaxSkipMissingData ( data , array ) ; if ( greyScale ) { ArrayByte result = replaceMissingValuesAndScale ( array , data , dataMinMax ) ; nextStart = geotiff . writeData ( ( byte [ ] ) result . getStorage ( ) , pageNumber ) ; } else { ArrayFloat result = replaceMissingValues ( array , data , dataMinMax ) ; nextStart = geotiff . writeData ( ( float [ ] ) result . getStorage ( ) , pageNumber ) ; } // set the width and the height int height = data . getShape ( ) [ 0 ] ; // Y int width = data . getShape ( ) [ 1 ] ; // X writeMetadata ( greyScale , xStart , yStart , xInc , yInc , height , width , pageNumber , nextStart , dataMinMax , proj ) ; pageNumber ++ ; }


Input: Search for an aliased coord that may have multiple variables : dimName = alias1 alias2 ; Variable alias1 ( dim ) ; Variable alias2 ( dim ) ; [CODESPLIT] private List < Variable > searchAliasedDimension ( NetcdfDataset ds , Dimension dim ) { String dimName = dim . getShortName ( ) ; String alias = ds . findAttValueIgnoreCase ( null , dimName , null ) ; if ( alias == null ) return null ; List < Variable > vars = new ArrayList <> ( ) ; StringTokenizer parser = new StringTokenizer ( alias , " ," ) ; while ( parser . hasMoreTokens ( ) ) { String token = parser . nextToken ( ) ; Variable ncvar = ds . findVariable ( token ) ; if ( ncvar == null ) continue ; if ( ncvar . getRank ( ) != 1 ) continue ; Iterator dimIter = ncvar . getDimensions ( ) . iterator ( ) ; Dimension dim2 = ( Dimension ) dimIter . next ( ) ; if ( dimName . equals ( dim2 . getShortName ( ) ) ) { vars . add ( ncvar ) ; if ( debug ) System . out . print ( " " + token ) ; } } if ( debug ) System . out . println ( ) ; return vars ; }


Input: Given the information on construction writes the necessary exception information . [CODESPLIT] public void write ( HttpServletResponse hsr ) throws IOException { PrintWriter xmlResponse = hsr . getWriter ( ) ; xmlResponse . append ( "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" ) ; xmlResponse . append ( "<ows:ExceptionReport xml:lang=\"en-US\" xsi:schemaLocation=\"http://www.opengis.net/ows/1.1" + " http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd\" version=\"2.0.0\" xmlns:ows=\"http://www.opengis.net/ows/1.1\"" + " xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">" ) ; xmlResponse . append ( "<ows:Exception " ) ; if ( locator != null ) xmlResponse . append ( "locator=\"" + locator + "\" " ) ; xmlResponse . append ( "exceptionCode=\"" + ExceptionCode + "\">" ) ; xmlResponse . append ( "<ows:ExceptionText>" + text + "</ows:ExceptionText>" ) ; xmlResponse . append ( "</ows:Exception>" ) ; xmlResponse . append ( "</ows:ExceptionReport>" ) ; }


Input: package private . use Array . factory () [CODESPLIT] static ArrayInt factory ( Index index , boolean isUnsigned ) { return ArrayInt . factory ( index , isUnsigned , null ) ; }


Input: / * create new ArrayInt with given indexImpl and backing store . Should be private . [CODESPLIT] static ArrayInt factory ( Index index , boolean isUnsigned , int [ ] storage ) { if ( index instanceof Index0D ) { return new ArrayInt . D0 ( index , isUnsigned , storage ) ; } else if ( index instanceof Index1D ) { return new ArrayInt . D1 ( index , isUnsigned , storage ) ; } else if ( index instanceof Index2D ) { return new ArrayInt . D2 ( index , isUnsigned , storage ) ; } else if ( index instanceof Index3D ) { return new ArrayInt . D3 ( index , isUnsigned , storage ) ; } else if ( index instanceof Index4D ) { return new ArrayInt . D4 ( index , isUnsigned , storage ) ; } else if ( index instanceof Index5D ) { return new ArrayInt . D5 ( index , isUnsigned , storage ) ; } else if ( index instanceof Index6D ) { return new ArrayInt . D6 ( index , isUnsigned , storage ) ; } else if ( index instanceof Index7D ) { return new ArrayInt . D7 ( index , isUnsigned , storage ) ; } else { return new ArrayInt ( index , isUnsigned , storage ) ; } }


Input: copy from javaArray to storage using the iterator : used by factory ( Object ) ; [CODESPLIT] protected void copyFrom1DJavaArray ( IndexIterator iter , Object javaArray ) { int [ ] ja = ( int [ ] ) javaArray ; for ( int aJa : ja ) iter . setIntNext ( aJa ) ; }


Input: package private : mostly for iterators [CODESPLIT] public double getDouble ( int index ) { int val = storage [ index ] ; return ( double ) ( isUnsigned ( ) ? DataType . unsignedIntToLong ( val ) : val ) ; }


Input: Begin API Override [CODESPLIT] @ Override public synchronized String nc_inq_libvers ( ) { String ret ; try { ce ( ) ; ret = nc4 . nc_inq_libvers ( ) ; if ( TRACE ) trace ( ret , "nc_inq_libvers" , "-" ) ; } finally { cx ( ) ; } return ret ; }


Input: open GribCollectionImmutable from an existing index file . return null on failure [CODESPLIT] static GribCollectionImmutable acquireGribCollection ( FileFactory factory , Object hashKey , String location , int buffer_size , CancelTask cancelTask , Object spiObject ) throws IOException { FileCacheable result ; DatasetUrl durl = new DatasetUrl ( null , location ) ; if ( gribCollectionCache != null ) { // FileFactory factory, Object hashKey, String location, int buffer_size, CancelTask cancelTask, Object spiObject result = GribCdmIndex . gribCollectionCache . acquire ( factory , hashKey , durl , buffer_size , cancelTask , spiObject ) ; } else { // String location, int buffer_size, ucar.nc2.util.CancelTask cancelTask, Object iospMessage result = factory . open ( durl , buffer_size , cancelTask , spiObject ) ; } return ( GribCollectionImmutable ) result ; }


Input: /////////////////////////////////////////////////////////////////////////////////////////////// [CODESPLIT] public static File getTopIndexFileFromConfig ( FeatureCollectionConfig config ) { File indexFile = makeTopIndexFileFromConfig ( config ) ; return GribIndexCache . getExistingFileOrCache ( indexFile . getPath ( ) ) ; }


Input: This is only used for the top level GribCollection . [CODESPLIT] private static File makeTopIndexFileFromConfig ( FeatureCollectionConfig config ) { Formatter errlog = new Formatter ( ) ; CollectionSpecParser specp = config . getCollectionSpecParser ( errlog ) ; String name = StringUtil2 . replace ( config . collectionName , ' ' , "/" ) ; // String cname = DirectoryCollection.makeCollectionName(name, Paths.get(specp.getRootDir())); return makeIndexFile ( name , new File ( specp . getRootDir ( ) ) ) ; }


Input: Find out what kind of index this is [CODESPLIT] public static GribCollectionType getType ( RandomAccessFile raf ) throws IOException { String magic ; raf . seek ( 0 ) ; magic = raf . readString ( Grib2CollectionWriter . MAGIC_START . getBytes ( CDM . utf8Charset ) . length ) ; switch ( magic ) { case Grib2CollectionWriter . MAGIC_START : return GribCollectionType . GRIB2 ; case Grib1CollectionWriter . MAGIC_START : return GribCollectionType . GRIB1 ; case Grib2PartitionBuilder . MAGIC_START : return GribCollectionType . Partition2 ; case Grib1PartitionBuilder . MAGIC_START : return GribCollectionType . Partition1 ; } return GribCollectionType . none ; }


Input: open GribCollectionImmutable from an existing index file . return null on failure [CODESPLIT] @ Nullable public static GribCollectionImmutable openCdmIndex ( String indexFilename , FeatureCollectionConfig config , boolean useCache , Logger logger ) throws IOException { File indexFileInCache = useCache ? GribIndexCache . getExistingFileOrCache ( indexFilename ) : new File ( indexFilename ) ; if ( indexFileInCache == null ) return null ; String indexFilenameInCache = indexFileInCache . getPath ( ) ; String name = makeNameFromIndexFilename ( indexFilename ) ; GribCollectionImmutable result = null ; try ( RandomAccessFile raf = RandomAccessFile . acquire ( indexFilenameInCache ) ) { GribCollectionType type = getType ( raf ) ; switch ( type ) { case GRIB2 : result = Grib2CollectionBuilderFromIndex . readFromIndex ( name , raf , config , logger ) ; break ; case Partition2 : result = Grib2PartitionBuilderFromIndex . createTimePartitionFromIndex ( name , raf , config , logger ) ; break ; case GRIB1 : result = Grib1CollectionBuilderFromIndex . readFromIndex ( name , raf , config , logger ) ; break ; case Partition1 : result = Grib1PartitionBuilderFromIndex . createTimePartitionFromIndex ( name , raf , config , logger ) ; break ; default : logger . warn ( "GribCdmIndex.openCdmIndex failed on {} type={}" , indexFilenameInCache , type ) ; } } catch ( FileNotFoundException ioe ) { throw ioe ; } catch ( Throwable t ) { logger . warn ( "GribCdmIndex.openCdmIndex failed on " + indexFilenameInCache , t ) ; RandomAccessFile . eject ( indexFilenameInCache ) ; if ( ! indexFileInCache . delete ( ) ) logger . warn ( "failed to delete {}" , indexFileInCache . getPath ( ) ) ; } return result ; }


Input: open GribCollectionImmutable from an existing index file . return null on failure [CODESPLIT] @ Nullable public static GribCollectionMutable openMutableGCFromIndex ( String indexFilename , FeatureCollectionConfig config , boolean dataOnly , boolean useCache , Logger logger ) { File indexFileInCache = useCache ? GribIndexCache . getExistingFileOrCache ( indexFilename ) : new File ( indexFilename ) ; if ( indexFileInCache == null ) { return null ; } String indexFilenameInCache = indexFileInCache . getPath ( ) ; String name = makeNameFromIndexFilename ( indexFilename ) ; GribCollectionMutable result = null ; try ( RandomAccessFile raf = RandomAccessFile . acquire ( indexFilenameInCache ) ) { GribCollectionType type = getType ( raf ) ; switch ( type ) { case GRIB2 : result = Grib2CollectionBuilderFromIndex . openMutableGCFromIndex ( name , raf , config , logger ) ; break ; case Partition2 : result = Grib2PartitionBuilderFromIndex . openMutablePCFromIndex ( name , raf , config , logger ) ; break ; case GRIB1 : result = Grib1CollectionBuilderFromIndex . openMutableGCFromIndex ( name , raf , config , logger ) ; break ; case Partition1 : result = Grib1PartitionBuilderFromIndex . openMutablePCFromIndex ( name , raf , config , logger ) ; break ; default : logger . warn ( "GribCdmIndex.openMutableGCFromIndex failed on {} type={}" , indexFilenameInCache , type ) ; } if ( result != null ) { result . lastModified = raf . getLastModified ( ) ; result . fileSize = raf . length ( ) ; } } catch ( Throwable t ) { logger . warn ( "GribCdmIndex.openMutableGCFromIndex failed on " + indexFilenameInCache , t ) ; } if ( result == null ) { RandomAccessFile . eject ( indexFilenameInCache ) ; if ( ! indexFileInCache . delete ( ) ) logger . warn ( "failed to delete {}" , indexFileInCache . getPath ( ) ) ; } return result ; }


Input: used ( only ) by GribCollectionBuilder [CODESPLIT] public static boolean updateGribCollectionFromPCollection ( boolean isGrib1 , PartitionManager dcm , CollectionUpdateType updateType , Formatter errlog , org . slf4j . Logger logger ) throws IOException { if ( updateType == CollectionUpdateType . never || dcm instanceof CollectionSingleIndexFile ) { // LOOK would isIndexFile() be better ? // then just open the existing index file return false ; } boolean changed = updatePartition ( isGrib1 , dcm , updateType , logger , errlog ) ; if ( errlog != null ) errlog . format ( "PartitionCollection %s was recreated %s%n" , dcm . getCollectionName ( ) , changed ) ; return changed ; }


Input: Update Grib Collection if needed [CODESPLIT] public static boolean updateGribCollection ( FeatureCollectionConfig config , CollectionUpdateType updateType , Logger logger ) throws IOException { if ( logger == null ) logger = classLogger ; long start = System . currentTimeMillis ( ) ; Formatter errlog = new Formatter ( ) ; CollectionSpecParser specp = config . getCollectionSpecParser ( errlog ) ; Path rootPath = Paths . get ( specp . getRootDir ( ) ) ; boolean isGrib1 = config . type == FeatureCollectionType . GRIB1 ; boolean changed ; if ( config . ptype == FeatureCollectionConfig . PartitionType . none || config . ptype == FeatureCollectionConfig . PartitionType . all ) { try ( CollectionAbstract dcm = new CollectionPathMatcher ( config , specp , logger ) ) { changed = updateGribCollection ( isGrib1 , dcm , updateType , FeatureCollectionConfig . PartitionType . none , logger , errlog ) ; } } else if ( config . ptype == FeatureCollectionConfig . PartitionType . timePeriod ) { try ( TimePartition tp = new TimePartition ( config , specp , logger ) ) { changed = updateTimePartition ( isGrib1 , tp , updateType , logger ) ; } } else { // LOOK assume wantSubdirs makes it into a Partition. Isnt there something better ?? if ( specp . wantSubdirs ( ) ) { // its a partition try ( DirectoryPartition dpart = new DirectoryPartition ( config , rootPath , true , new GribCdmIndex ( logger ) , NCX_SUFFIX , logger ) ) { dpart . putAuxInfo ( FeatureCollectionConfig . AUX_CONFIG , config ) ; changed = updateDirectoryCollectionRecurse ( isGrib1 , dpart , config , updateType , logger ) ; } } else { // otherwise its a leaf directory changed = updateLeafCollection ( isGrib1 , config , updateType , true , logger , rootPath ) ; } } long took = System . currentTimeMillis ( ) - start ; logger . info ( "updateGribCollection {} changed {} took {} msecs" , config . collectionName , changed , took ) ; return changed ; }


Input: return true if changed exception on failure [CODESPLIT] public static boolean updateGribCollection ( boolean isGrib1 , MCollection dcm , CollectionUpdateType updateType , FeatureCollectionConfig . PartitionType ptype , Logger logger , Formatter errlog ) throws IOException { logger . debug ( "GribCdmIndex.updateGribCollection %s %s%n" , dcm . getCollectionName ( ) , updateType ) ; if ( ! isUpdateNeeded ( dcm . getIndexFilename ( NCX_SUFFIX ) , updateType , ( isGrib1 ? GribCollectionType . GRIB1 : GribCollectionType . GRIB2 ) , logger ) ) return false ; boolean changed ; if ( isGrib1 ) { // existing case handles correctly - make seperate index for each runtime (OR) partition == runtime Grib1CollectionBuilder builder = new Grib1CollectionBuilder ( dcm . getCollectionName ( ) , dcm , logger ) ; changed = builder . updateNeeded ( updateType ) && builder . createIndex ( ptype , errlog ) ; } else { Grib2CollectionBuilder builder = new Grib2CollectionBuilder ( dcm . getCollectionName ( ) , dcm , logger ) ; changed = builder . updateNeeded ( updateType ) && builder . createIndex ( ptype , errlog ) ; } return changed ; }


Input: return true if changed exception on failure [CODESPLIT] private static boolean updatePartition ( boolean isGrib1 , PartitionManager dcm , CollectionUpdateType updateType , Logger logger , Formatter errlog ) throws IOException { boolean changed ; if ( isGrib1 ) { Grib1PartitionBuilder builder = new Grib1PartitionBuilder ( dcm . getCollectionName ( ) , new File ( dcm . getRoot ( ) ) , dcm , logger ) ; changed = builder . updateNeeded ( updateType ) && builder . createPartitionedIndex ( updateType , errlog ) ; } else { Grib2PartitionBuilder builder = new Grib2PartitionBuilder ( dcm . getCollectionName ( ) , new File ( dcm . getRoot ( ) ) , dcm , logger ) ; changed = builder . updateNeeded ( updateType ) && builder . createPartitionedIndex ( updateType , errlog ) ; } return changed ; }


Input: Update all the gbx indices in one directory and the ncx index for that directory [CODESPLIT] private static boolean updateLeafCollection ( boolean isGrib1 , FeatureCollectionConfig config , CollectionUpdateType updateType , boolean isTop , Logger logger , Path dirPath ) throws IOException { if ( config . ptype == FeatureCollectionConfig . PartitionType . file ) { return updateFilePartition ( isGrib1 , config , updateType , isTop , logger , dirPath ) ; } else { Formatter errlog = new Formatter ( ) ; CollectionSpecParser specp = config . getCollectionSpecParser ( errlog ) ; try ( DirectoryCollection dcm = new DirectoryCollection ( config . collectionName , dirPath , isTop , config . olderThan , logger ) ) { dcm . putAuxInfo ( FeatureCollectionConfig . AUX_CONFIG , config ) ; if ( specp . getFilter ( ) != null ) dcm . setStreamFilter ( new StreamFilter ( specp . getFilter ( ) , specp . getFilterOnName ( ) ) ) ; boolean changed = updateGribCollection ( isGrib1 , dcm , updateType , FeatureCollectionConfig . PartitionType . directory , logger , errlog ) ; logger . debug ( "  GribCdmIndex.updateDirectoryPartition was updated=%s on %s%n" , changed , dirPath ) ; return changed ; } } }


Input: File Partition : each File is a collection of Grib records and the collection of all files in the directory is a PartitionCollection . Rewrite the PartitionCollection and optionally its children [CODESPLIT] private static boolean updateFilePartition ( final boolean isGrib1 , final FeatureCollectionConfig config , final CollectionUpdateType updateType , boolean isTop , final Logger logger , Path dirPath ) throws IOException { long start = System . currentTimeMillis ( ) ; final Formatter errlog = new Formatter ( ) ; CollectionSpecParser specp = config . getCollectionSpecParser ( errlog ) ; try ( FilePartition partition = new FilePartition ( config . collectionName , dirPath , isTop , config . olderThan , logger ) ) { partition . putAuxInfo ( FeatureCollectionConfig . AUX_CONFIG , config ) ; if ( specp . getFilter ( ) != null ) partition . setStreamFilter ( new StreamFilter ( specp . getFilter ( ) , specp . getFilterOnName ( ) ) ) ; logger . debug ( "GribCdmIndex.updateFilePartition %s %s%n" , partition . getCollectionName ( ) , updateType ) ; if ( ! isUpdateNeeded ( partition . getIndexFilename ( NCX_SUFFIX ) , updateType , ( isGrib1 ? GribCollectionType . Partition1 : GribCollectionType . Partition2 ) , logger ) ) return false ; final AtomicBoolean anyChange = new AtomicBoolean ( false ) ; // just need a mutable boolean we can declare final // redo the children here if ( updateType != CollectionUpdateType . testIndexOnly ) { // skip children on testIndexOnly partition . iterateOverMFileCollection ( mfile -> { MCollection part = new CollectionSingleFile ( mfile , logger ) ; part . putAuxInfo ( FeatureCollectionConfig . AUX_CONFIG , config ) ; try { boolean changed = updateGribCollection ( isGrib1 , part , updateType , FeatureCollectionConfig . PartitionType . file , logger , errlog ) ; if ( changed ) anyChange . set ( true ) ; } catch ( IllegalStateException t ) { logger . warn ( "Error making partition {} '{}'" , part . getRoot ( ) , t . getMessage ( ) ) ; partition . removePartition ( part ) ; // keep on truckin; can happen if directory is empty } catch ( Throwable t ) { logger . error ( "Error making partition " + part . getRoot ( ) , t ) ; partition . removePartition ( part ) ; } } ) ; } // LOOK what if theres only one file? try { // redo partition index if needed, will detect if children have changed boolean recreated = updatePartition ( isGrib1 , partition , updateType , logger , errlog ) ; long took = System . currentTimeMillis ( ) - start ; if ( recreated ) logger . info ( "RewriteFilePartition {} took {} msecs" , partition . getCollectionName ( ) , took ) ; return recreated ; } catch ( IllegalStateException t ) { logger . warn ( "Error making partition {} '{}'" , partition . getRoot ( ) , t . getMessage ( ) ) ; return false ; } catch ( Throwable t ) { logger . error ( "Error making partition " + partition . getRoot ( ) , t ) ; return false ; } } }


Input: Open GribCollection from config . CollectionUpdater calls InvDatasetFc . update () calls InvDatasetFcGrib . updateCollection () [CODESPLIT] public static GribCollectionImmutable openGribCollection ( FeatureCollectionConfig config , CollectionUpdateType updateType , Logger logger ) throws IOException { // update if needed boolean changed = updateGribCollection ( config , updateType , logger ) ; File idxFile = makeTopIndexFileFromConfig ( config ) ; // If call to updateGribCollection shows a change happened, then collection changed. // If updateType is never (tds is in charge of updating, not TDM or some other external application), // then this is being called after receiving an outside trigger. Assume collection changed. // // At this point, there isn't a good way of invalidating the gribColectionCache entries associated with the // particular collection being updated, so we have to clear the whole cache. Will revisit this in // 5.0 if performance is an issue if ( ( updateType == CollectionUpdateType . never ) || changed ) { gribCollectionCache . clearCache ( true ) ; } return openCdmIndex ( idxFile . getPath ( ) , config , true , logger ) ; }


Input: Used by IOSPs [CODESPLIT] public static GribCollectionImmutable openGribCollectionFromRaf ( RandomAccessFile raf , FeatureCollectionConfig config , CollectionUpdateType updateType , org . slf4j . Logger logger ) throws IOException { GribCollectionImmutable result ; // check if its a plain ole GRIB1/2 data file boolean isGrib1 = false ; boolean isGrib2 = Grib2RecordScanner . isValidFile ( raf ) ; if ( ! isGrib2 ) isGrib1 = Grib1RecordScanner . isValidFile ( raf ) ; if ( isGrib1 || isGrib2 ) { result = openGribCollectionFromDataFile ( isGrib1 , raf , config , updateType , null , logger ) ; // close the data file, the ncx raf file is managed by gribCollection raf . close ( ) ; } else { // check its an ncx file result = openGribCollectionFromIndexFile ( raf , config , logger ) ; } return result ; }


Input: Open a grib collection from a single grib1 or grib2 file . Create the gbx9 and ncx2 files if needed . [CODESPLIT] private static GribCollectionImmutable openGribCollectionFromDataFile ( boolean isGrib1 , RandomAccessFile dataRaf , FeatureCollectionConfig config , CollectionUpdateType updateType , Formatter errlog , org . slf4j . Logger logger ) throws IOException { String filename = dataRaf . getLocation ( ) ; File dataFile = new File ( filename ) ; MFile mfile = new MFileOS ( dataFile ) ; return openGribCollectionFromDataFile ( isGrib1 , mfile , updateType , config , errlog , logger ) ; }


Input: from a single file read in the index create if it doesnt exist ; return null on failure [CODESPLIT] @ Nullable public static GribCollectionImmutable openGribCollectionFromDataFile ( boolean isGrib1 , MFile mfile , CollectionUpdateType updateType , FeatureCollectionConfig config , Formatter errlog , org . slf4j . Logger logger ) throws IOException { MCollection dcm = new CollectionSingleFile ( mfile , logger ) ; dcm . putAuxInfo ( FeatureCollectionConfig . AUX_CONFIG , config ) ; if ( isGrib1 ) { Grib1CollectionBuilder builder = new Grib1CollectionBuilder ( dcm . getCollectionName ( ) , dcm , logger ) ; // LOOK ignoring partition type boolean changed = ( builder . updateNeeded ( updateType ) && builder . createIndex ( FeatureCollectionConfig . PartitionType . file , errlog ) ) ; } else { Grib2CollectionBuilder builder = new Grib2CollectionBuilder ( dcm . getCollectionName ( ) , dcm , logger ) ; boolean changed = ( builder . updateNeeded ( updateType ) && builder . createIndex ( FeatureCollectionConfig . PartitionType . file , errlog ) ) ; } // the index file should now exist, open it GribCollectionImmutable result = openCdmIndex ( dcm . getIndexFilename ( NCX_SUFFIX ) , config , true , logger ) ; if ( result != null ) return result ; // if open fails, force recreate the index if ( updateType == CollectionUpdateType . never ) return null ; // not allowed to write if ( updateType == CollectionUpdateType . always ) return null ; // already tried to force write, give up return openGribCollectionFromDataFile ( isGrib1 , mfile , CollectionUpdateType . always , config , errlog , logger ) ; }


Input: Create a grib collection / partition collection from an existing ncx2 file . PartionCollection . partition . getGribCollection () . [CODESPLIT] @ Nullable public static GribCollectionImmutable openGribCollectionFromIndexFile ( RandomAccessFile indexRaf , FeatureCollectionConfig config , org . slf4j . Logger logger ) throws IOException { GribCollectionType type = getType ( indexRaf ) ; String location = indexRaf . getLocation ( ) ; File f = new File ( location ) ; int pos = f . getName ( ) . lastIndexOf ( "." ) ; String name = ( pos > 0 ) ? f . getName ( ) . substring ( 0 , pos ) : f . getName ( ) ; // remove ".ncx2" switch ( type ) { case Partition1 : return Grib1PartitionBuilderFromIndex . createTimePartitionFromIndex ( name , indexRaf , config , logger ) ; case GRIB1 : return Grib1CollectionBuilderFromIndex . readFromIndex ( name , indexRaf , config , logger ) ; case Partition2 : return Grib2PartitionBuilderFromIndex . createTimePartitionFromIndex ( name , indexRaf , config , logger ) ; case GRIB2 : return Grib2CollectionBuilderFromIndex . readFromIndex ( name , indexRaf , config , logger ) ; } return null ; }


Input: / IndexReader interface [CODESPLIT] @ Override public boolean readChildren ( Path indexFile , AddChildCallback callback ) throws IOException { logger . debug ( "GribCdmIndex.readChildren %s%n" , indexFile ) ; try ( RandomAccessFile raf = RandomAccessFile . acquire ( indexFile . toString ( ) ) ) { GribCollectionType type = getType ( raf ) ; if ( type == GribCollectionType . Partition1 || type == GribCollectionType . Partition2 ) { if ( openIndex ( raf , logger ) ) { String topDir = gribCollectionIndex . getTopDir ( ) ; int n = gribCollectionIndex . getMfilesCount ( ) ; // partition index files stored in MFiles for ( int i = 0 ; i < n ; i ++ ) { GribCollectionProto . MFile mfilep = gribCollectionIndex . getMfiles ( i ) ; callback . addChild ( topDir , mfilep . getFilename ( ) , mfilep . getLastModified ( ) ) ; } return true ; } } return false ; } }


Input: Match has different semantics than urlCompare [CODESPLIT] static boolean urlMatch ( URL pattern , URL url ) { int relation ; if ( pattern == null ) return ( url == null ) ; if ( ! ( url . getHost ( ) . endsWith ( pattern . getHost ( ) ) ) ) return false ; // e.g. pattern=x.y.org url=y.org if ( ! ( url . getPath ( ) . startsWith ( pattern . getPath ( ) ) ) ) return false ; // e.g. pattern=y.org/a/b url=y.org/a if ( pattern . getPort ( ) > 0 && pattern . getPort ( ) != url . getPort ( ) ) return false ; // note: all other fields are ignored return true ; }


Input: Allow users to add to the default rc [CODESPLIT] static synchronized public void add ( String key , String value , String url ) { if ( key == null ) return ; if ( ! initialized ) RC . initialize ( ) ; Triple t = new Triple ( key , value , url ) ; dfaltRC . insert ( t ) ; // recompute well-knowns setWellKnown ( ) ; }


Input: Allow users to search the default rc [CODESPLIT] static synchronized public String find ( String key , String url ) { if ( key == null ) return null ; if ( ! initialized ) RC . initialize ( ) ; Triple t = dfaltRC . lookup ( key , url ) ; return ( t == null ? null : t . value ) ; }


Input: Record some well known parameters [CODESPLIT] static void setWellKnown ( ) { if ( dfaltRC . triplestore . size ( ) == 0 ) return ; // Walk the set of triples looking for those that have no url for ( String key : dfaltRC . keySet ( ) ) { Triple triple = dfaltRC . lookup ( key ) ; if ( triple . url == null ) { RC . set ( key , triple . value ) ; // let set sort it out } } }


Input: overwrite existing entries [CODESPLIT] public boolean load ( String abspath ) { abspath = abspath . replace ( ' ' , ' ' ) ; File rcFile = new File ( abspath ) ; if ( ! rcFile . exists ( ) || ! rcFile . canRead ( ) ) { return false ; } if ( showlog ) log . debug ( "Loading rc file: " + abspath ) ; try ( BufferedReader rdr = new BufferedReader ( new InputStreamReader ( new FileInputStream ( rcFile ) , CDM . UTF8 ) ) ) { for ( int lineno = 1 ; ; lineno ++ ) { URL url = null ; String line = rdr . readLine ( ) ; if ( line == null ) break ; // trim leading blanks line = line . trim ( ) ; if ( line . length ( ) == 0 ) continue ; // empty line if ( line . charAt ( 0 ) == ' ' ) continue ; // check for comment // parse the line if ( line . charAt ( 0 ) == LTAG ) { int rindex = line . indexOf ( RTAG ) ; if ( rindex < 0 ) return false ; if ( showlog ) log . error ( "Malformed [url] at " + abspath + "." + lineno ) ; String surl = line . substring ( 1 , rindex ) ; try { url = new URL ( surl ) ; } catch ( MalformedURLException mue ) { if ( showlog ) log . error ( "Malformed [url] at " + abspath + "." + lineno ) ; } line = line . substring ( rindex + 1 ) ; // trim again line = line . trim ( ) ; } // Get the key,value part String [ ] pieces = line . split ( "\\s*=\\s*" ) ; assert ( pieces . length == 1 || pieces . length == 2 ) ; // Create the triple String value = "1" ; if ( pieces . length == 2 ) value = pieces [ 1 ] . trim ( ) ; Triple triple = new Triple ( pieces [ 0 ] . trim ( ) , value , url ) ; List < Triple > list = triplestore . get ( triple . key ) ; if ( list == null ) list = new ArrayList < Triple > ( ) ; Triple prev = addtriple ( list , triple ) ; triplestore . put ( triple . key , list ) ; } } catch ( FileNotFoundException fe ) { if ( showlog ) log . debug ( "Loading rc file: " + abspath ) ; return false ; } catch ( IOException ioe ) { if ( showlog ) log . error ( "File " + abspath + ": IO exception: " + ioe . getMessage ( ) ) ; return false ; } return true ; }


Input: Allow for external loading [CODESPLIT] public Triple insert ( Triple t ) { if ( t . key == null ) return null ; List < Triple > list = triplestore . get ( t . key ) ; if ( list == null ) list = new ArrayList < Triple > ( ) ; Triple prev = addtriple ( list , t ) ; triplestore . put ( t . key , list ) ; return prev ; }


Input: Prints the original string representation of this clause . For use in debugging . [CODESPLIT] public void printConstraint ( PrintWriter os ) { lhs . printConstraint ( os ) ; String op = ExprParserConstants . tokenImage [ operator ] ; op = op . substring ( 1 , op . length ( ) - 1 ) ; os . print ( op ) ; //os.print(ExprParserConstants.tokenImage[operator].substring(2, 3)); if ( rhs . size ( ) == 1 ) { ( ( ValueClause ) rhs . get ( 0 ) ) . printConstraint ( os ) ; } else { os . print ( "{" ) ; Iterator it = rhs . iterator ( ) ; boolean first = true ; while ( it . hasNext ( ) ) { if ( ! first ) os . print ( "," ) ; ( ( ValueClause ) it . next ( ) ) . printConstraint ( os ) ; first = false ; } os . print ( "}" ) ; } }


Input: Get top level datasets contained directly in this catalog . Do not dereference catRefs . [CODESPLIT] public List < Dataset > getDatasetsLocal ( ) { List < Dataset > datasets = ( List < Dataset > ) flds . get ( Dataset . Datasets ) ; return datasets == null ? new ArrayList <> ( 0 ) : datasets ; }


Input: Look though all datasets here or under here . do not go into catrefs [CODESPLIT] public Dataset findDatasetByName ( String name ) { for ( Dataset ds : getDatasets ( ) ) { if ( ds . getName ( ) . equals ( name ) ) return ds ; Dataset result = ds . findDatasetByName ( name ) ; if ( result != null ) return result ; } return null ; }


Input: Sets new projection for subsequent drawing . [CODESPLIT] public void setProjection ( ProjectionImpl project ) { displayProject = project ; if ( featSetList == null ) return ; Iterator iter = featSetList . iterator ( ) ; while ( iter . hasNext ( ) ) { FeatureSet fs = ( FeatureSet ) iter . next ( ) ; fs . newProjection = true ; } }


Input: we have to deal with both projections and resolution - dependence [CODESPLIT] protected Iterator getShapes ( java . awt . Graphics2D g , AffineTransform normal2device ) { long startTime = System . currentTimeMillis ( ) ; if ( featSetList == null ) { initFeatSetList ( ) ; assert ! featSetList . isEmpty ( ) ; } // which featureSet should we ue? FeatureSet fs = ( FeatureSet ) featSetList . get ( 0 ) ; if ( featSetList . size ( ) > 1 ) { // compute scale double scale = 1.0 ; try { AffineTransform world2device = g . getTransform ( ) ; AffineTransform world2normal = normal2device . createInverse ( ) ; world2normal . concatenate ( world2device ) ; scale = Math . max ( Math . abs ( world2normal . getScaleX ( ) ) , Math . abs ( world2normal . getShearX ( ) ) ) ; // drawing or printing if ( Debug . isSet ( "GisFeature/showTransform" ) ) { System . out . println ( "GisFeature/showTransform: " + world2normal + "\n scale = " + scale ) ; } } catch ( java . awt . geom . NoninvertibleTransformException e ) { System . out . println ( " GisRenderFeature: NoninvertibleTransformException on " + normal2device ) ; } if ( ! displayProject . isLatLon ( ) ) scale *= 111.0 ; // km/deg double minD = Double . MAX_VALUE ; for ( Object aFeatSetList : featSetList ) { FeatureSet tryfs = ( FeatureSet ) aFeatSetList ; double d = Math . abs ( scale * tryfs . minDist - pixelMatch ) ; // we want min features ~ 2 pixels if ( d < minD ) { minD = d ; fs = tryfs ; } } if ( Debug . isSet ( "GisFeature/MapResolution" ) ) { System . out . println ( "GisFeature/MapResolution: scale = " + scale + " minDist = " + fs . minDist ) ; } } // we may have deferred the actual creation of the points if ( fs . featureList == null ) fs . createFeatures ( ) ; // ok, now see if we need to project if ( ! displayProject . equals ( fs . project ) ) { fs . setProjection ( displayProject ) ; } else { // deal with LatLon if ( fs . newProjection && displayProject . isLatLon ( ) ) { fs . setProjection ( displayProject ) ; } } fs . newProjection = false ; if ( Debug . isSet ( "GisFeature/timing/getShapes" ) ) { long tookTime = System . currentTimeMillis ( ) - startTime ; System . out . println ( "timing.getShapes: " + tookTime * .001 + " seconds" ) ; } // so return it, already return fs . getShapes ( ) ; }


Input:
make an ArrayList of Shapes from the given featureList and current display Projection [CODESPLIT] private ArrayList makeShapes ( Iterator featList ) { Shape shape ; ArrayList shapeList = new ArrayList ( ) ; ProjectionImpl dataProject = getDataProjection ( ) ; if ( Debug . isSet ( "GisFeature/MapDraw" ) ) { System . out . println ( "GisFeature/MapDraw: makeShapes with " + displayProject ) ; } /*    if (Debug.isSet("bug.drawShapes")) {
      int count =0;
      // make each GisPart a seperate shape for debugging
feats:while (featList.hasNext()) {
        AbstractGisFeature feature = (AbstractGisFeature) featList.next();
        java.util.Iterator pi = feature.getGisParts();
        while (pi.hasNext()) {
          GisPart gp = (GisPart) pi.next();
          int np = gp.getNumPoints();
          GeneralPath path = new GeneralPath(GeneralPath.WIND_EVEN_ODD, np);
          double[] xx = gp.getX();
          double[] yy = gp.getY();
          path.moveTo((float) xx[0], (float) yy[0]);
          if (count == 63)
                System.out.println("moveTo x ="+xx[0]+" y= "+yy[0]);
          for(int i = 1; i < np; i++) {
            path.lineTo((float) xx[i], (float) yy[i]);
            if (count == 63)
                System.out.println("lineTo x ="+xx[i]+" y= "+yy[i]);
          }
          shapeList.add(path);
          if (count == 63)
            break feats;
          count++;
        }
      }
      System.out.println("bug.drawShapes: #shapes =" +shapeList.size());
      return shapeList;
    }  */ while ( featList . hasNext ( ) ) { AbstractGisFeature feature = ( AbstractGisFeature ) featList . next ( ) ; if ( dataProject . isLatLon ( ) ) // always got to run it through if its lat/lon shape = feature . getProjectedShape ( displayProject ) ; else if ( dataProject == displayProject ) shape = feature . getShape ( ) ; else shape = feature . getProjectedShape ( dataProject , displayProject ) ; shapeList . add ( shape ) ; } return shapeList ; }


Input: Compares this PrefixName with a string . [CODESPLIT] public final int compareTo ( String string ) { return getID ( ) . length ( ) >= string . length ( ) ? getID ( ) . compareToIgnoreCase ( string ) : getID ( ) . compareToIgnoreCase ( string . substring ( 0 , getID ( ) . length ( ) ) ) ; }


Input: The given task is run in a background thread . Progress is indicated once a second . You cannot call this method again till the task is completed . [CODESPLIT] public boolean startProgressMonitorTask ( ProgressMonitorTask pmt ) { if ( busy ) return false ; busy = true ; this . task = pmt ; isCancelled = false ; count = 0 ; setIcon ( icon [ 0 ] ) ; // create timer, whose events happen on the awt event Thread ActionListener watcher = new ActionListener ( ) { public void actionPerformed ( ActionEvent evt ) { //System.out.println("timer event"+evt); if ( isCancelled && ! task . isCancel ( ) ) { task . cancel ( ) ; if ( debug ) System . out . println ( " task.cancel" ) ; return ; // give it a chance to finish up } else { // indicate progress count ++ ; setIcon ( icon [ count % 2 ] ) ; if ( debug ) System . out . println ( " stop count=" + count ) ; } // need to make sure task acknowledges the cancel; so dont shut down // until the task is done if ( task . isDone ( ) ) { if ( myTimer != null ) myTimer . stop ( ) ; myTimer = null ; if ( task . isError ( ) ) javax . swing . JOptionPane . showMessageDialog ( null , task . getErrorMessage ( ) ) ; if ( task . isSuccess ( ) ) fireEvent ( new ActionEvent ( this , 0 , "success" ) ) ; else if ( task . isError ( ) ) fireEvent ( new ActionEvent ( this , 0 , "error" ) ) ; else if ( task . isCancel ( ) ) fireEvent ( new ActionEvent ( this , 0 , "cancel" ) ) ; else fireEvent ( new ActionEvent ( this , 0 , "done" ) ) ; busy = false ; } } } ; myTimer = new javax . swing . Timer ( 1000 , watcher ) ; // every second myTimer . start ( ) ; // do task in a seperate, non-event, thread Thread taskThread = new Thread ( task ) ; taskThread . start ( ) ; return true ; }


Input: Update all the grib indices in one directory and the collection index for that directory [CODESPLIT] private Accum scanLeafDirectoryCollection ( boolean isGrib1 , FeatureCollectionConfig config , Counters parentCounters , Logger logger , Path dirPath , boolean isTop , Indent indent , Formatter fm ) throws IOException { if ( config . ptype == FeatureCollectionConfig . PartitionType . file ) { reportOneFileHeader ( indent , fm ) ; fm . format ( "%sDirectory %s%n" , indent , dirPath ) ; } Accum accum = new Accum ( ) ; int nfiles = 0 ; Counters countersThisDir = parentCounters . makeSubCounters ( ) ; Formatter errlog = new Formatter ( ) ; CollectionSpecParser specp = config . getCollectionSpecParser ( errlog ) ; DirectoryCollection dcm = new DirectoryCollection ( config . collectionName , dirPath , isTop , config . olderThan , logger ) ; // dcm.setUseGribFilter(false); dcm . putAuxInfo ( FeatureCollectionConfig . AUX_CONFIG , config ) ; if ( specp . getFilter ( ) != null ) dcm . setStreamFilter ( new StreamFilter ( specp . getFilter ( ) , specp . getFilterOnName ( ) ) ) ; try ( CloseableIterator < MFile > iter = dcm . getFileIterator ( ) ) { while ( iter . hasNext ( ) ) { MFile mfile = iter . next ( ) ; Counters countersOneFile = countersThisDir . makeSubCounters ( ) ; int nrecords = 0 ; if ( isGrib1 ) { Grib1Index grib1Index = readGrib1Index ( mfile , false ) ; if ( grib1Index == null ) { System . out . printf ( "%s%s: read or create failed%n" , indent , mfile . getPath ( ) ) ; continue ; } for ( ucar . nc2 . grib . grib1 . Grib1Record gr : grib1Index . getRecords ( ) ) { accumGrib1Record ( gr , countersOneFile ) ; nrecords ++ ; } } else { Grib2Index grib2Index = readGrib2Index ( mfile , false ) ; if ( grib2Index == null ) { System . out . printf ( "%s%s: read or create failed%n" , indent , mfile . getPath ( ) ) ; continue ; } for ( ucar . nc2 . grib . grib2 . Grib2Record gr : grib2Index . getRecords ( ) ) { accumGrib2Record ( gr , countersOneFile ) ; nrecords ++ ; } } accum . nrecords += nrecords ; countersThisDir . addTo ( countersOneFile ) ; if ( config . ptype == FeatureCollectionConfig . PartitionType . file ) reportOneFile ( mfile , nrecords , countersOneFile , indent , fm ) ; nfiles ++ ; // get file sizes String path = mfile . getPath ( ) ; if ( path . endsWith ( GribIndex . GBX9_IDX ) ) { accum . indexSize += ( ( float ) mfile . getLength ( ) / ( 1000 * 1000 ) ) ; // mb } else { accum . fileSize += ( ( float ) mfile . getLength ( ) / ( 1000 * 1000 ) ) ; // mb File idxFile = GribIndexCache . getExistingFileOrCache ( path + GribIndex . GBX9_IDX ) ; if ( idxFile . exists ( ) ) accum . indexSize += ( ( float ) idxFile . length ( ) / ( 1000 * 1000 ) ) ; // mb } } } parentCounters . addTo ( countersThisDir ) ; accum . nfiles += nfiles ; accum . last = reportOneDir ( dirPath . toString ( ) , accum , countersThisDir , indent , accum . last ) ; return accum ; }


Input: LOOK need an option to only scan latest last partition or something [CODESPLIT] private boolean needsUpdate ( CollectionUpdateType ff , File collectionIndexFile ) throws IOException { long collectionLastModified = collectionIndexFile . lastModified ( ) ; Set < String > newFileSet = new HashSet <> ( ) ; for ( MCollection dcm : partitionManager . makePartitions ( CollectionUpdateType . test ) ) { String partitionIndexFilename = StringUtil2 . replace ( dcm . getIndexFilename ( GribCdmIndex . NCX_SUFFIX ) , ' ' , "/" ) ; File partitionIndexFile = GribIndexCache . getExistingFileOrCache ( partitionIndexFilename ) ; if ( partitionIndexFile == null ) // make sure each partition has an index return true ; if ( collectionLastModified < partitionIndexFile . lastModified ( ) ) // and the partition index is earlier than the collection index return true ; newFileSet . add ( partitionIndexFilename ) ; } if ( ff == CollectionUpdateType . testIndexOnly ) return false ; // now see if any files were deleted GribCdmIndex reader = new GribCdmIndex ( logger ) ; List < MFile > oldFiles = new ArrayList <> ( ) ; reader . readMFiles ( collectionIndexFile . toPath ( ) , oldFiles ) ; Set < String > oldFileSet = new HashSet <> ( ) ; for ( MFile oldFile : oldFiles ) { if ( ! newFileSet . contains ( oldFile . getPath ( ) ) ) return true ; // got deleted - must recreate the index oldFileSet . add ( oldFile . getPath ( ) ) ; } // now see if any files were added for ( String newFilename : newFileSet ) { if ( ! oldFileSet . contains ( newFilename ) ) return true ; // got added - must recreate the index } return false ; }


Input: return true if changed exception on failure [CODESPLIT] boolean createPartitionedIndex ( CollectionUpdateType forcePartition , Formatter errlog ) throws IOException { if ( errlog == null ) errlog = new Formatter ( ) ; // info will be discarded // create partitions from the partitionManager for ( MCollection dcmp : partitionManager . makePartitions ( forcePartition ) ) { dcmp . putAuxInfo ( FeatureCollectionConfig . AUX_CONFIG , partitionManager . getAuxInfo ( FeatureCollectionConfig . AUX_CONFIG ) ) ; result . addPartition ( dcmp ) ; } result . sortPartitions ( ) ; // after this the partition list is immutable // choose the "canonical" partition, aka prototype // only used in copyInfo int n = result . getPartitionSize ( ) ; if ( n == 0 ) { errlog . format ( "ERR Nothing in this partition = %s%n" , result . showLocation ( ) ) ; throw new IllegalStateException ( "Nothing in this partition =" + result . showLocation ( ) ) ; } int idx = partitionManager . getProtoIndex ( n ) ; PartitionCollectionMutable . Partition canon = result . getPartition ( idx ) ; logger . debug ( "     Using canonical partition {}" , canon . getDcm ( ) . getCollectionName ( ) ) ; try ( GribCollectionMutable gc = canon . makeGribCollection ( ) ) { // LOOK open/close canonical partition if ( gc == null ) throw new IllegalStateException ( "canon.makeGribCollection failed on =" + result . showLocation ( ) + " " + canon . getName ( ) + "; errs=" + errlog ) ; // copy info from canon gribCollection to result partitionCollection result . copyInfo ( gc ) ; result . isPartitionOfPartitions = ( gc instanceof PartitionCollectionMutable ) ; result . dateRange = gc . dateRange ; } // check consistency across vert and ens coords // create partitioned variables // partition index is used - do not resort partitions GribCollectionMutable . Dataset ds2D = makeDataset2D ( errlog ) ; if ( ds2D == null ) { errlog . format ( " ERR makeDataset2D failed, index not written on %s%n" , result . showLocation ( ) ) ; throw new IllegalStateException ( "makeDataset2D failed, index not written on =" + result . showLocation ( ) + "; errs=" + errlog ) ; } // Make Best for a TwoD if ( ds2D . gctype == GribCollectionImmutable . Type . TwoD ) makeDatasetBest ( ds2D , false ) ; //else if (ds2D.gctype == GribCollectionImmutable.Type.MRUTC) //  makeTime2runtime(ds2D, false); // ready to write the index file return writeIndex ( result , errlog ) ; }


Input: / * LOOK heres a place where one could post process and combine instead of at coverage level . this would benefit iosp ie the netcdf API private void makeTime2runtime ( GribCollectionMutable . Dataset ds2D boolean isComplete ) throws IOException { [CODESPLIT] private void makeDatasetBest ( GribCollectionMutable . Dataset ds2D , boolean isComplete ) { GribCollectionMutable . Dataset dsBest = result . makeDataset ( isComplete ? GribCollectionImmutable . Type . BestComplete : GribCollectionImmutable . Type . Best ) ; int npart = result . getPartitionSize ( ) ; // for each 2D group for ( GribCollectionMutable . GroupGC group2D : ds2D . groups ) { GribCollectionMutable . GroupGC groupB = dsBest . addGroupCopy ( group2D ) ; // make copy of group, add to Best dataset groupB . isTwoD = false ; // for each time2D, create the best time coordinates HashMap < Coordinate , CoordinateTimeAbstract > map2DtoBest = new HashMap <> ( ) ; // associate 2D coord with best CoordinateSharerBest sharer = new CoordinateSharerBest ( ) ; for ( Coordinate coord : group2D . coords ) { if ( coord instanceof CoordinateRuntime ) continue ; // skip it if ( coord instanceof CoordinateTime2D ) { CoordinateTimeAbstract best = ( ( CoordinateTime2D ) coord ) . makeBestTimeCoordinate ( result . masterRuntime ) ; if ( ! isComplete ) best = best . makeBestFromComplete ( ) ; sharer . addCoordinate ( best ) ; map2DtoBest . put ( coord , best ) ; } else { sharer . addCoordinate ( coord ) ; } } groupB . coords = sharer . finish ( ) ; // these are the unique coords for group Best // transfer variables to Best group, set shared Coordinates for ( GribCollectionMutable . VariableIndex vi2d : group2D . variList ) { // copy vi2d and add to groupB PartitionCollectionMutable . VariableIndexPartitioned vip = result . makeVariableIndexPartitioned ( groupB , vi2d , npart ) ; vip . finish ( ) ; // set shared coordinates List < Coordinate > newCoords = new ArrayList <> ( ) ; for ( Integer groupIndex : vi2d . coordIndex ) { Coordinate coord2D = group2D . coords . get ( groupIndex ) ; if ( coord2D instanceof CoordinateRuntime ) continue ; // skip runtime; if ( coord2D instanceof CoordinateTime2D ) { newCoords . add ( map2DtoBest . get ( coord2D ) ) ; // add the best coordinate for that CoordinateTime2D } else { newCoords . add ( coord2D ) ; } } vip . coordIndex = sharer . reindex ( newCoords ) ; } } // loop over groups }


Input:
/ * MAGIC_START version sizeRecords VariableRecords ( sizeRecords bytes ) sizeIndex GribCollectionIndex ( sizeIndex bytes ) [CODESPLIT] protected boolean writeIndex ( PartitionCollectionMutable pc , Formatter f ) throws IOException { File idxFile = GribIndexCache . getFileOrCache ( partitionManager . getIndexFilename ( GribCdmIndex . NCX_SUFFIX ) ) ; if ( idxFile . exists ( ) ) { RandomAccessFile . eject ( idxFile . getPath ( ) ) ; if ( ! idxFile . delete ( ) ) logger . error ( "gc2tp cant delete " + idxFile . getPath ( ) ) ; } writer = new GribCollectionWriter ( null , null ) ; try ( RandomAccessFile raf = new RandomAccessFile ( idxFile . getPath ( ) , "rw" ) ) { raf . order ( RandomAccessFile . BIG_ENDIAN ) ; //// header message raf . write ( getMagicStart ( ) . getBytes ( CDM . utf8Charset ) ) ; raf . writeInt ( getVersion ( ) ) ; raf . writeLong ( 0 ) ; // no record section /*
      message GribCollection {
        required string name = 1;         // must be unique - index filename is name.ncx
        required string topDir = 2;       // filenames are reletive to this
        repeated MFile mfiles = 3;        // list of grib MFiles
        repeated Dataset dataset = 4;
        repeated Gds gds = 5;             // unique Gds, shared amongst datasets

        required int32 center = 6;      // these 4 fields are to get a GribTable object
        required int32 subcenter = 7;
        required int32 master = 8;
        required int32 local = 9;       // grib1 table Version

        optional int32 genProcessType = 10;
        optional int32 genProcessId = 11;
        optional int32 backProcessId = 12;

        // repeated Parameter params = 20;      // not used
        FcConfig config = 21;
        uint64 startTime = 22; // calendar date, first valid time
        uint64 endTime = 23;   // calendar date, last valid time

        extensions 100 to 199;
      }

      extend GribCollection {
        repeated Partition partitions = 100;
      }
       */ GribCollectionProto . GribCollection . Builder indexBuilder = GribCollectionProto . GribCollection . newBuilder ( ) ; indexBuilder . setName ( pc . getName ( ) ) ; Path topDir = pc . directory . toPath ( ) ; String pathS = StringUtil2 . replace ( topDir . toString ( ) , ' ' , "/" ) ; indexBuilder . setTopDir ( pathS ) ; // mfiles are the partition indexes int count = 0 ; for ( PartitionCollectionMutable . Partition part : pc . partitions ) { GribCollectionProto . MFile . Builder b = GribCollectionProto . MFile . newBuilder ( ) ; String pathRS = makeReletiveFilename ( pc , part ) ; // reletive to pc.directory b . setFilename ( pathRS ) ; b . setLastModified ( part . getLastModified ( ) ) ; b . setLength ( part . fileSize ) ; b . setIndex ( count ++ ) ; indexBuilder . addMfiles ( b . build ( ) ) ; } indexBuilder . setCenter ( pc . center ) ; indexBuilder . setSubcenter ( pc . subcenter ) ; indexBuilder . setMaster ( pc . master ) ; indexBuilder . setLocal ( pc . local ) ; indexBuilder . setGenProcessId ( pc . genProcessId ) ; indexBuilder . setGenProcessType ( pc . genProcessType ) ; indexBuilder . setBackProcessId ( pc . backProcessId ) ; indexBuilder . setStartTime ( pc . dateRange . getStart ( ) . getMillis ( ) ) ; indexBuilder . setEndTime ( pc . dateRange . getEnd ( ) . getMillis ( ) ) ; indexBuilder . setMasterRuntime ( writer . writeCoordProto ( pc . masterRuntime ) ) ; // dataset for ( GribCollectionMutable . Dataset ds : pc . datasets ) indexBuilder . addDataset ( writeDatasetProto ( pc , ds ) ) ; // extensions if ( pc . run2part != null ) { for ( int part : pc . run2part ) indexBuilder . addRun2Part ( part ) ; } for ( PartitionCollectionMutable . Partition part : pc . partitions ) indexBuilder . addPartitions ( writePartitionProto ( pc , part ) ) ; indexBuilder . setIsPartitionOfPartitions ( pc . isPartitionOfPartitions ) ; // write it out GribCollectionProto . GribCollection index = indexBuilder . build ( ) ; byte [ ] b = index . toByteArray ( ) ; NcStream . writeVInt ( raf , b . length ) ; // message size raf . write ( b ) ; // message  - all in one gulp f . format ( "Grib2PartitionIndex= %d bytes file size =  %d bytes%n%n" , b . length , raf . length ( ) ) ; } return true ; }


Input: / * message Dataset { required Type type = 1 ; repeated Group groups = 2 ; } [CODESPLIT] private GribCollectionProto . Dataset writeDatasetProto ( PartitionCollectionMutable pc , GribCollectionMutable . Dataset ds ) throws IOException { GribCollectionProto . Dataset . Builder b = GribCollectionProto . Dataset . newBuilder ( ) ; GribCollectionProto . Dataset . Type type = GribCollectionProto . Dataset . Type . valueOf ( ds . gctype . toString ( ) ) ; b . setType ( type ) ; for ( GribCollectionMutable . GroupGC group : ds . groups ) b . addGroups ( writeGroupProto ( pc , group ) ) ; return b . build ( ) ; }


Input: / * message Group { Gds gds = 1 ; // use this to build the HorizCoordSys repeated Variable variables = 2 ; // list of variables repeated Coord coords = 3 ; // list of coordinates repeated uint32 fileno = 4 [ packed = true ] ; // the component files that are in this group key into gc . mfiles } [CODESPLIT] private GribCollectionProto . Group writeGroupProto ( PartitionCollectionMutable pc , GribCollectionMutable . GroupGC g ) throws IOException { GribCollectionProto . Group . Builder b = GribCollectionProto . Group . newBuilder ( ) ; b . setGds ( GribCollectionWriter . writeGdsProto ( g . horizCoordSys . getRawGds ( ) , g . horizCoordSys . getPredefinedGridDefinition ( ) ) ) ; for ( GribCollectionMutable . VariableIndex vb : g . variList ) { b . addVariables ( writeVariableProto ( ( PartitionCollectionMutable . VariableIndexPartitioned ) vb ) ) ; } for ( Coordinate coord : g . coords ) { switch ( coord . getType ( ) ) { case runtime : b . addCoords ( writer . writeCoordProto ( ( CoordinateRuntime ) coord ) ) ; break ; case time : b . addCoords ( writer . writeCoordProto ( ( CoordinateTime ) coord ) ) ; break ; case timeIntv : b . addCoords ( writer . writeCoordProto ( ( CoordinateTimeIntv ) coord ) ) ; break ; case time2D : b . addCoords ( writer . writeCoordProto ( ( CoordinateTime2D ) coord ) ) ; break ; case vert : b . addCoords ( writer . writeCoordProto ( ( CoordinateVert ) coord ) ) ; break ; case ens : b . addCoords ( writer . writeCoordProto ( ( CoordinateEns ) coord ) ) ; break ; } } if ( g . filenose != null ) for ( Integer fileno : g . filenose ) b . addFileno ( fileno ) ; return b . build ( ) ; }


Input:
/ * message Variable { uint32 discipline = 1 ; bytes pds = 2 ; // raw pds repeated uint32 ids = 3 [ packed = true ] ; // extra info not in pds ; grib2 id section [CODESPLIT] private GribCollectionProto . Variable writeVariableProto ( PartitionCollectionMutable . VariableIndexPartitioned vp ) { GribCollectionProto . Variable . Builder b = GribCollectionProto . Variable . newBuilder ( ) ; b . setDiscipline ( vp . discipline ) ; b . setPds ( ByteString . copyFrom ( vp . rawPds ) ) ; // extra id info b . addIds ( vp . center ) ; b . addIds ( vp . subcenter ) ; b . setRecordsPos ( vp . recordsPos ) ; b . setRecordsLen ( vp . recordsLen ) ; for ( int idx : vp . coordIndex ) b . addCoordIdx ( idx ) ; b . setNdups ( vp . ndups ) ; b . setNrecords ( vp . nrecords ) ; b . setMissing ( vp . nmissing ) ; /* if (vp.twot != null) { // only for 2D
      for (int invCount : vp.twot.getCount())
        b.addInvCount(invCount);
    }

    if (vp.time2runtime != null) { // only for 1D
      for (int idx=0; idx < vp.time2runtime.getN(); idx++)
        b.addTime2Runtime(vp.time2runtime.get(idx));
    } */ // extensions if ( vp . nparts > 0 && vp . partnoSA != null ) { for ( int i = 0 ; i < vp . nparts ; i ++ ) // PartitionCollection.PartitionForVariable2D pvar : vp.getPartitionForVariable2D()) b . addPartVariable ( writePartitionVariableProto ( vp . partnoSA . get ( i ) , vp . groupnoSA . get ( i ) , vp . varnoSA . get ( i ) , vp . nrecords , vp . ndups , vp . nmissing ) ) ; // LOOK was it finished ?? } return b . build ( ) ; }


Input: / * message PartitionVariable { uint32 groupno = 1 ; uint32 varno = 2 ; uint32 flag = 3 ; uint32 partno = 4 ; [CODESPLIT] private GribCollectionProto . PartitionVariable writePartitionVariableProto ( int partno , int groupno , int varno , int nrecords , int ndups , int nmissing ) { GribCollectionProto . PartitionVariable . Builder pb = GribCollectionProto . PartitionVariable . newBuilder ( ) ; pb . setPartno ( partno ) ; pb . setGroupno ( groupno ) ; pb . setVarno ( varno ) ; pb . setNdups ( ndups ) ; pb . setNrecords ( nrecords ) ; pb . setMissing ( nmissing ) ; return pb . build ( ) ; }


Input: / * message Partition { string name = 1 ; // name is used in TDS - eg the subdirectory when generated by TimePartitionCollections string filename = 2 ; // the gribCollection . ncx file reletive to gc . string directory = 3 ; // top directory NOT USED uint64 lastModified = 4 ; int64 length = 5 ; int64 partitionDate = 6 ; // partition date added 11 / 25 / 14 } [CODESPLIT] private GribCollectionProto . Partition writePartitionProto ( PartitionCollectionMutable pc , PartitionCollectionMutable . Partition p ) { GribCollectionProto . Partition . Builder b = GribCollectionProto . Partition . newBuilder ( ) ; String pathRS = makeReletiveFilename ( pc , p ) ; // reletive to pc.directory b . setFilename ( pathRS ) ; b . setName ( p . name ) ; // b.setDirectory(p.directory); b . setLastModified ( p . lastModified ) ; b . setLength ( p . fileSize ) ; if ( p . partitionDate != null ) b . setPartitionDate ( p . partitionDate . getMillis ( ) ) ; // LOOK what about calendar ?? return b . build ( ) ; }


Input: LOOK [CODESPLIT] public boolean equalsData ( ucar . nc2 . ft . fmrc . EnsCoord other ) { if ( ensembles != other . ensembles ) return false ; if ( pdn != other . pdn ) return false ; for ( int i = 0 ; i < ensTypes . length ; i ++ ) { if ( ensTypes [ i ] != other . ensTypes [ i ] ) return false ; } return true ; }


Input:
////////////// [CODESPLIT] static public EnsCoord findEnsCoord ( List < EnsCoord > ensCoords , EnsCoord want ) { if ( want == null ) return null ; for ( EnsCoord ec : ensCoords ) { if ( want . equalsData ( ec ) ) return ec ; } // make a new one
 EnsCoord result = new EnsCoord ( want ) ; ensCoords . add ( result ) ; return result ; }


Input:
Extend result with all the values in the list of EnsCoord [CODESPLIT] static public void normalize ( EnsCoord result , List < EnsCoord > ecList ) { List < EnsCoord > extra = new ArrayList <> ( ) ; for ( EnsCoord ec : ecList ) { if ( ! result . equalsData ( ec ) ) { // differences can only be greater
 extra . add ( ec ) ; } } if ( extra . size ( ) == 0 ) return ; for ( EnsCoord ec : extra ) { if ( ec . getNEnsembles ( ) < result . getNEnsembles ( ) ) continue ; result = ec ; } }


Input: / * message Record { uint32 fileno = 1 ; // which GRIB file ? key into GC . fileMap uint64 pos = 2 ; // offset in GRIB file of the start of entire message uint64 bmsPos = 3 ; // use alternate bms if non - zero ( grib2 only ) uint32 drsOffset = 4 ; // offset of drs from pos ( grib2 only ) } [CODESPLIT] private GribCollectionProto . SparseArray writeSparseArray ( Grib1CollectionBuilder . VariableBag vb , Set < Integer > fileSet ) { GribCollectionProto . SparseArray . Builder b = GribCollectionProto . SparseArray . newBuilder ( ) ; SparseArray < Grib1Record > sa = vb . coordND . getSparseArray ( ) ; for ( int size : sa . getShape ( ) ) { b . addSize ( size ) ; } for ( int track : sa . getTrack ( ) ) { b . addTrack ( track ) ; } for ( Grib1Record gr : sa . getContent ( ) ) { GribCollectionProto . Record . Builder br = GribCollectionProto . Record . newBuilder ( ) ; br . setFileno ( gr . getFile ( ) ) ; fileSet . add ( gr . getFile ( ) ) ; Grib1SectionIndicator is = gr . getIs ( ) ; br . setStartPos ( is . getStartPos ( ) ) ; // start of entire message b . addRecords ( br ) ; } b . setNdups ( sa . getNdups ( ) ) ; return b . build ( ) ; }


Input: / * message Dataset { Type type = 1 ; repeated Group groups = 2 ; [CODESPLIT] private GribCollectionProto . Dataset writeDatasetProto ( GribCollectionImmutable . Type type , List < Group > groups ) { GribCollectionProto . Dataset . Builder b = GribCollectionProto . Dataset . newBuilder ( ) ; GribCollectionProto . Dataset . Type ptype = GribCollectionProto . Dataset . Type . valueOf ( type . toString ( ) ) ; b . setType ( ptype ) ; for ( Group group : groups ) { b . addGroups ( writeGroupProto ( group ) ) ; } return b . build ( ) ; }


Input: / * message Group { Gds gds = 1 ; // use this to build the HorizCoordSys repeated Variable variables = 2 ; // list of variables repeated Coord coords = 3 ; // list of coordinates repeated int32 fileno = 4 [ packed = true ] ; // the component files that are in this group key into gc . mfiles } [CODESPLIT] private GribCollectionProto . Group writeGroupProto ( Group g ) { GribCollectionProto . Group . Builder b = GribCollectionProto . Group . newBuilder ( ) ; b . setGds ( writeGdsProto ( g . gdss . getRawBytes ( ) , g . gdss . getPredefinedGridDefinition ( ) ) ) ; for ( Grib1CollectionBuilder . VariableBag vbag : g . gribVars ) { b . addVariables ( writeVariableProto ( vbag ) ) ; } for ( Coordinate coord : g . coords ) { switch ( coord . getType ( ) ) { case runtime : b . addCoords ( writeCoordProto ( ( CoordinateRuntime ) coord ) ) ; break ; case time : b . addCoords ( writeCoordProto ( ( CoordinateTime ) coord ) ) ; break ; case timeIntv : b . addCoords ( writeCoordProto ( ( CoordinateTimeIntv ) coord ) ) ; break ; case time2D : b . addCoords ( writeCoordProto ( ( CoordinateTime2D ) coord ) ) ; break ; case vert : b . addCoords ( writeCoordProto ( ( CoordinateVert ) coord ) ) ; break ; case ens : b . addCoords ( writeCoordProto ( ( CoordinateEns ) coord ) ) ; break ; } } for ( Integer aFileSet : g . fileSet ) { b . addFileno ( aFileSet ) ; } return b . build ( ) ; }


Input: / * message Variable { uint32 discipline = 1 ; bytes pds = 2 ; // raw pds repeated uint32 ids = 3 [ packed = true ] ; // extra info not in pds ; grib2 id section [CODESPLIT] private GribCollectionProto . Variable writeVariableProto ( Grib1CollectionBuilder . VariableBag vb ) { GribCollectionProto . Variable . Builder b = GribCollectionProto . Variable . newBuilder ( ) ; b . setDiscipline ( 0 ) ; b . setPds ( ByteString . copyFrom ( vb . first . getPDSsection ( ) . getRawBytes ( ) ) ) ; b . setRecordsPos ( vb . pos ) ; b . setRecordsLen ( vb . length ) ; for ( int idx : vb . coordIndex ) { b . addCoordIdx ( idx ) ; } // keep stats SparseArray sa = vb . coordND . getSparseArray ( ) ; if ( sa != null ) { b . setNdups ( sa . getNdups ( ) ) ; b . setNrecords ( sa . countNotMissing ( ) ) ; b . setMissing ( sa . countMissing ( ) ) ; } return b . build ( ) ; }


Input: Set the index - th StructureData of this ArrayStructure . [CODESPLIT] public void setObject ( int index , Object value ) { if ( sdata == null ) sdata = new StructureData [ nelems ] ; sdata [ index ] = ( StructureData ) value ; }


Input: Get the index - th StructureData of this ArrayStructure . [CODESPLIT] public StructureData getStructureData ( int index ) { if ( sdata == null ) sdata = new StructureData [ nelems ] ; if ( index >= sdata . length ) throw new IllegalArgumentException ( index + " > " + sdata . length ) ; if ( sdata [ index ] == null ) sdata [ index ] = makeStructureData ( this , index ) ; return sdata [ index ] ; }


Input: Get member data of any type for a specific record as an Array . This may avoid the overhead of creating the StructureData object but is equivilent to getStructure ( recno ) . getArray ( Member m ) . [CODESPLIT] public Array getArray ( int recno , StructureMembers . Member m ) { DataType dataType = m . getDataType ( ) ; switch ( dataType ) { case DOUBLE : double [ ] da = getJavaArrayDouble ( recno , m ) ; return Array . factory ( dataType , m . getShape ( ) , da ) ; case FLOAT : float [ ] fa = getJavaArrayFloat ( recno , m ) ; return Array . factory ( dataType , m . getShape ( ) , fa ) ; case BYTE : case UBYTE : case ENUM1 : byte [ ] ba = getJavaArrayByte ( recno , m ) ; return Array . factory ( dataType , m . getShape ( ) , ba ) ; case SHORT : case USHORT : case ENUM2 : short [ ] sa = getJavaArrayShort ( recno , m ) ; return Array . factory ( dataType , m . getShape ( ) , sa ) ; case INT : case UINT : case ENUM4 : int [ ] ia = getJavaArrayInt ( recno , m ) ; return Array . factory ( dataType , m . getShape ( ) , ia ) ; case ULONG : case LONG : long [ ] la = getJavaArrayLong ( recno , m ) ; return Array . factory ( dataType , m . getShape ( ) , la ) ; case CHAR : char [ ] ca = getJavaArrayChar ( recno , m ) ; return Array . factory ( dataType , m . getShape ( ) , ca ) ; case STRING : String [ ] str = getJavaArrayString ( recno , m ) ; return Array . factory ( dataType , m . getShape ( ) , str ) ; case STRUCTURE : return getArrayStructure ( recno , m ) ; case SEQUENCE : return getArraySequence ( recno , m ) ; case OPAQUE : return getArrayObject ( recno , m ) ; } throw new RuntimeException ( "Dont have implemenation for " + dataType ) ; }


Input:
Set data for one member over all structures . This is used by VariableDS to do scale / offset . [CODESPLIT] public void setMemberArray ( StructureMembers . Member m , Array memberArray ) { m . setDataArray ( memberArray ) ; if ( memberArray instanceof ArrayStructure ) { // LOOK
 ArrayStructure as = ( ArrayStructure ) memberArray ; m . setStructureMembers ( as . getStructureMembers ( ) ) ; } }