Input: 根据指定类型解析json字符串，并返回该类型的对象 [CODESPLIT] public static < T > T fromJson ( String jsonString , Class < T > clazz ) { if ( StringHelper . isEmpty ( jsonString ) ) { return null ; } try { return mapper . readValue ( jsonString , clazz ) ; } catch ( Exception e ) { log . warn ( "parse json string error:" + jsonString , e ) ; return null ; } }


Input: 根据key获取配置的字符串value值 [CODESPLIT] public static String getProperty ( String key ) { if ( key == null ) { return null ; } return getProperties ( ) . getProperty ( key ) ; }


Input: 根据key获取配置的数字value值 [CODESPLIT] public static int getNumerProperty ( String key ) { String value = getProperties ( ) . getProperty ( key ) ; if ( NumberUtils . isNumber ( value ) ) { return Integer . parseInt ( value ) ; } else { return 0 ; } }


Input: 根据指定的文件名称，从类路径中加载属性文件，构造Properties对象 [CODESPLIT] public static void loadProperties ( String filename ) { InputStream in = null ; ClassLoader threadContextClassLoader = Thread . currentThread ( ) . getContextClassLoader ( ) ; properties = new Properties ( ) ; if ( threadContextClassLoader != null ) { in = threadContextClassLoader . getResourceAsStream ( filename ) ; } if ( in == null ) { in = ConfigHelper . class . getResourceAsStream ( filename ) ; if ( in == null ) { log . warn ( "No properties file found in the classpath by filename " + filename ) ; } } else { try { properties . load ( in ) ; log . info ( "Properties read " + properties ) ; } catch ( Exception e ) { log . error ( "Error reading from " + filename , e ) ; } finally { try { in . close ( ) ; } catch ( IOException e ) { log . warn ( "IOException while closing InputStream: " + e . getMessage ( ) ) ; } } } }


Input: Checks whether no more bytes will be returned . [CODESPLIT] protected boolean noMoreCharacters ( ) throws IOException { if ( avail == 0 ) { avail = is . read ( buffer ) ; if ( avail <= 0 ) { avail = 0 ; return true ; } pos = 0 ; } return false ; }


Input: Reads a line into the given byte array . [CODESPLIT] public int readLine ( final byte [ ] array , final EnumSet < LineTerminator > terminators ) throws IOException { return readLine ( array , 0 , array . length , terminators ) ; }


Input: Reads a line into the given byte - array fragment using { @linkplain #ALL_TERMINATORS all terminators } . [CODESPLIT] public int readLine ( final byte [ ] array , final int off , final int len ) throws IOException { return readLine ( array , off , len , ALL_TERMINATORS ) ; }


Input: Reads a line into the given byte - array fragment . [CODESPLIT] public int readLine ( final byte [ ] array , final int off , final int len , final EnumSet < LineTerminator > terminators ) throws IOException { ByteArrays . ensureOffsetLength ( array , off , len ) ; if ( len == 0 ) return 0 ; // 0-length reads always return 0 if ( noMoreCharacters ( ) ) return - 1 ; int i , k = 0 , remaining = len , read = 0 ; // The number of bytes still to be read for ( ; ; ) { for ( i = 0 ; i < avail && i < remaining && ( k = buffer [ pos + i ] ) != ' ' && k != ' ' ; i ++ ) ; System . arraycopy ( buffer , pos , array , off + read , i ) ; pos += i ; avail -= i ; read += i ; remaining -= i ; if ( remaining == 0 ) { readBytes += read ; return read ; // We did not stop because of a terminator } if ( avail > 0 ) { // We met a terminator if ( k == ' ' ) { // LF first pos ++ ; avail -- ; if ( terminators . contains ( LineTerminator . LF ) ) { readBytes += read + 1 ; return read ; } else { array [ off + read ++ ] = ' ' ; remaining -- ; } } else if ( k == ' ' ) { // CR first pos ++ ; avail -- ; if ( terminators . contains ( LineTerminator . CR_LF ) ) { if ( avail > 0 ) { if ( buffer [ pos ] == ' ' ) { // CR/LF with LF already in the buffer. pos ++ ; avail -- ; readBytes += read + 2 ; return read ; } } else { // We must search for the LF. if ( noMoreCharacters ( ) ) { // Not found a matching LF because of end of file, will return CR in buffer if not a terminator if ( ! terminators . contains ( LineTerminator . CR ) ) { array [ off + read ++ ] = ' ' ; remaining -- ; readBytes += read ; } else readBytes += read + 1 ; return read ; } if ( buffer [ 0 ] == ' ' ) { // Found matching LF, won't return terminators in the buffer pos ++ ; avail -- ; readBytes += read + 2 ; return read ; } } } if ( terminators . contains ( LineTerminator . CR ) ) { readBytes += read + 1 ; return read ; } array [ off + read ++ ] = ' ' ; remaining -- ; } } else if ( noMoreCharacters ( ) ) { readBytes += read ; return read ; } } }


Input: Returns the length of the underlying input stream if it is { @linkplain MeasurableStream measurable } . [CODESPLIT] @ Override public long length ( ) throws IOException { if ( measurableStream != null ) return measurableStream . length ( ) ; if ( fileChannel != null ) return fileChannel . size ( ) ; throw new UnsupportedOperationException ( ) ; }


Input: Skips the given amount of bytes by repeated reads . [CODESPLIT] private long skipByReading ( final long n ) throws IOException { long toSkip = n ; int len ; while ( toSkip > 0 ) { len = is . read ( buffer , 0 , ( int ) Math . min ( buffer . length , toSkip ) ) ; if ( len > 0 ) toSkip -= len ; else break ; } return n - toSkip ; }


Input: Skips over and discards the given number of bytes of data from this fast buffered input stream . [CODESPLIT] @ Override public long skip ( final long n ) throws IOException { if ( n <= avail ) { final int m = ( int ) n ; pos += m ; avail -= m ; readBytes += n ; return n ; } long toSkip = n - avail , result = 0 ; avail = 0 ; while ( toSkip != 0 && ( result = is == System . in ? skipByReading ( toSkip ) : is . skip ( toSkip ) ) < toSkip ) { if ( result == 0 ) { if ( is . read ( ) == - 1 ) break ; toSkip -- ; } else toSkip -= result ; } final long t = n - ( toSkip - result ) ; readBytes += t ; return t ; }


Input: Ensures that a range given by its first ( inclusive ) and last ( exclusive ) elements fits an array of given length . [CODESPLIT] public static void ensureFromTo ( final int arrayLength , final int from , final int to ) { if ( from < 0 ) throw new ArrayIndexOutOfBoundsException ( "Start index (" + from + ") is negative" ) ; if ( from > to ) throw new IllegalArgumentException ( "Start index (" + from + ") is greater than end index (" + to + ")" ) ; if ( to > arrayLength ) throw new ArrayIndexOutOfBoundsException ( "End index (" + to + ") is greater than array length (" + arrayLength + ")" ) ; }


Input: Ensures that a range given by an offset and a length fits an array of given length . [CODESPLIT] public static void ensureOffsetLength ( final int arrayLength , final int offset , final int length ) { if ( offset < 0 ) throw new ArrayIndexOutOfBoundsException ( "Offset (" + offset + ") is negative" ) ; if ( length < 0 ) throw new IllegalArgumentException ( "Length (" + length + ") is negative" ) ; if ( offset + length > arrayLength ) throw new ArrayIndexOutOfBoundsException ( "Last index (" + ( offset + length ) + ") is greater than array length (" + arrayLength + ")" ) ; }


Input: Transforms two consecutive sorted ranges into a single sorted range . The initial ranges are { [CODESPLIT] private static void inPlaceMerge ( final int from , int mid , final int to , final IntComparator comp , final Swapper swapper ) { if ( from >= mid || mid >= to ) return ; if ( to - from == 2 ) { if ( comp . compare ( mid , from ) < 0 ) swapper . swap ( from , mid ) ; return ; } int firstCut ; int secondCut ; if ( mid - from > to - mid ) { firstCut = from + ( mid - from ) / 2 ; secondCut = lowerBound ( mid , to , firstCut , comp ) ; } else { secondCut = mid + ( to - mid ) / 2 ; firstCut = upperBound ( from , mid , secondCut , comp ) ; } int first2 = firstCut ; int middle2 = mid ; int last2 = secondCut ; if ( middle2 != first2 && middle2 != last2 ) { int first1 = first2 ; int last1 = middle2 ; while ( first1 < -- last1 ) swapper . swap ( first1 ++ , last1 ) ; first1 = middle2 ; last1 = last2 ; while ( first1 < -- last1 ) swapper . swap ( first1 ++ , last1 ) ; first1 = first2 ; last1 = last2 ; while ( first1 < -- last1 ) swapper . swap ( first1 ++ , last1 ) ; } mid = firstCut + ( secondCut - mid ) ; inPlaceMerge ( from , firstCut , mid , comp , swapper ) ; inPlaceMerge ( mid , secondCut , to , comp , swapper ) ; }


Input: Performs a binary search on an already - sorted range : finds the first position where an element can be inserted without violating the ordering . Sorting is by a user - supplied comparison function . [CODESPLIT] private static int lowerBound ( int from , final int to , final int pos , final IntComparator comp ) { // if (comp==null) throw new NullPointerException(); int len = to - from ; while ( len > 0 ) { int half = len / 2 ; int middle = from + half ; if ( comp . compare ( middle , pos ) < 0 ) { from = middle + 1 ; len -= half + 1 ; } else { len = half ; } } return from ; }


Input: Performs a binary search on an already sorted range : finds the last position where an element can be inserted without violating the ordering . Sorting is by a user - supplied comparison function . [CODESPLIT] private static int upperBound ( int from , final int mid , final int pos , final IntComparator comp ) { // if (comp==null) throw new NullPointerException(); int len = mid - from ; while ( len > 0 ) { int half = len / 2 ; int middle = from + half ; if ( comp . compare ( pos , middle ) < 0 ) { len = half ; } else { from = middle + 1 ; len -= half + 1 ; } } return from ; }


Input:
Sorts the specified range of elements using the specified swapper and according to the order induced by the specified comparator using mergesort . [CODESPLIT] public static void mergeSort ( final int from , final int to , final IntComparator c , final Swapper swapper ) { /*
		 * We retain the same method signature as quickSort. Given only a comparator and swapper we
		 * do not know how to copy and move elements from/to temporary arrays. Hence, in contrast to
		 * the JDK mergesorts this is an "in-place" mergesort, i.e. does not allocate any temporary
		 * arrays. A non-inplace mergesort would perhaps be faster in most cases, but would require
		 * non-intuitive delegate objects...
		 */ final int length = to - from ; // Insertion sort on smallest arrays if ( length < MERGESORT_NO_REC ) { for ( int i = from ; i < to ; i ++ ) { for ( int j = i ; j > from && ( c . compare ( j - 1 , j ) > 0 ) ; j -- ) { swapper . swap ( j , j - 1 ) ; } } return ; } // Recursively sort halves int mid = ( from + to ) >>> 1 ; mergeSort ( from , mid , c , swapper ) ; mergeSort ( mid , to , c , swapper ) ; // If list is already sorted, nothing left to do. This is an // optimization that results in faster sorts for nearly ordered lists. if ( c . compare ( mid - 1 , mid ) <= 0 ) return ; // Merge sorted halves inPlaceMerge ( from , mid , to , c , swapper ) ; }


Input: Swaps two sequences of elements using a provided swapper . [CODESPLIT] protected static void swap ( final Swapper swapper , int a , int b , final int n ) { for ( int i = 0 ; i < n ; i ++ , a ++ , b ++ ) swapper . swap ( , b ) ; }


Input: Sorts the specified range of elements using the specified swapper and according to the order induced by the specified comparator using a parallel quicksort . [CODESPLIT] public static void parallelQuickSort ( final int from , final int to , final IntComparator comp , final Swapper swapper ) { final ForkJoinPool pool = new ForkJoinPool ( Runtime . getRuntime ( ) . availableProcessors ( ) ) ; pool . invoke ( new ForkJoinGenericQuickSort ( from , to , comp , swapper ) ) ; pool . shutdown ( ) ; }


Input: Sorts the specified range of elements using the specified swapper and according to the order induced by the specified comparator using parallel quicksort . [CODESPLIT] public static void quickSort ( final int from , final int to , final IntComparator comp , final Swapper swapper ) { final int len = to - from ; // Insertion sort on smallest arrays if ( len < QUICKSORT_NO_REC ) { for ( int i = from ; i < to ; i ++ ) for ( int j = i ; j > from && ( comp . compare ( j - 1 , j ) > 0 ) ; j -- ) { swapper . swap ( j , j - 1 ) ; } return ; } // Choose a partition element, v int m = from + len / 2 ; // Small arrays, middle element int l = from ; int n = to - 1 ; if ( len > QUICKSORT_MEDIAN_OF_9 ) { // Big arrays, pseudomedian of 9 int s = len / 8 ; l = med3 ( l , l + s , l + 2 * s , comp ) ; m = med3 ( m - s , m , m + s , comp ) ; n = med3 ( n - 2 * s , n - s , n , comp ) ; } m = med3 ( l , m , n , comp ) ; // Mid-size, med of 3 // int v = x[m]; int a = from ; int b = a ; int c = to - 1 ; // Establish Invariant: v* (<v)* (>v)* v* int d = c ; while ( true ) { int comparison ; while ( b <= c && ( ( comparison = comp . compare ( b , m ) ) <= 0 ) ) { if ( comparison == 0 ) { // Fix reference to pivot if necessary if ( a == m ) m = b ; else if ( b == m ) m = a ; swapper . swap ( a ++ , b ) ; } b ++ ; } while ( c >= b && ( ( comparison = comp . compare ( c , m ) ) >= 0 ) ) { if ( comparison == 0 ) { // Fix reference to pivot if necessary if ( c == m ) m = d ; else if ( d == m ) m = c ; swapper . swap ( c , d -- ) ; } c -- ; } if ( b > c ) break ; // Fix reference to pivot if necessary if ( b == m ) m = d ; else if ( c == m ) m = c ; swapper . swap ( b ++ , c -- ) ; } // Swap partition elements back to middle int s ; s = Math . min ( a - from , b - a ) ; swap ( swapper , from , b - s , s ) ; s = Math . min ( d - c , to - d - 1 ) ; swap ( swapper , b , to - s , s ) ; // Recursively sort non-partition-elements if ( ( s = b - a ) > 1 ) quickSort ( from , from + s , comp , swapper ) ; if ( ( s = d - c ) > 1 ) quickSort ( to - s , to , comp , swapper ) ; }


Input: Avalanches the bits of an integer by applying the finalisation step of MurmurHash3 . [CODESPLIT] public static int murmurHash3 ( int x ) { x ^= x >>> 16 ; x *= 0x85ebca6b ; x ^= x >>> 13 ; x *= 0xc2b2ae35 ; x ^= x >>> 16 ; return x ; }


Input: Avalanches the bits of a long integer by applying the finalisation step of MurmurHash3 . [CODESPLIT] public static long murmurHash3 ( long x ) { x ^= x >>> 33 ; x *= 0xff51afd7ed558ccd  L ; x ^= x >>> 33 ; x *= 0xc4ceb9fe1a85ec53  L ; x ^= x >>> 33 ; return x ; }


Input: Clears the content of this { [CODESPLIT] public void clear ( ) throws IOException { if ( ! fileChannel . isOpen ( ) ) throw new IOException ( "This " + getClass ( ) . getSimpleName ( ) + " is closed" ) ; writePosition = position = inspectable = 0 ; mark = - 1 ; }


Input: Appends the content of a specified buffer to the end of the currently represented stream . [CODESPLIT] @ Override public int write ( final ByteBuffer byteBuffer ) throws IOException { ensureOpen ( ) ; final int remaining = byteBuffer . remaining ( ) ; if ( inspectable < buffer . length ) { // Still some space in the inspectable buffer. final int toBuffer = Math . min ( buffer . length - inspectable , remaining ) ; byteBuffer . get ( buffer , inspectable , toBuffer ) ; inspectable += toBuffer ; } if ( byteBuffer . hasRemaining ( ) ) { fileChannel . position ( writePosition ) ; writePosition += fileChannel . write ( byteBuffer ) ; } return remaining ; }


Input: Truncates the overflow file to a given size if possible . [CODESPLIT] public void truncate ( final long size ) throws FileNotFoundException , IOException { fileChannel . truncate ( Math . max ( size , writePosition ) ) ; }


Input: Reads bytes from this byte - array input stream as specified in { [CODESPLIT] @ Override public int read ( final byte b [ ] , final int offset , final int length ) { if ( this . length == this . position ) return length == 0 ? 0 : - 1 ; final int n = Math . min ( length , this . length - this . position ) ; System . arraycopy ( array , this . offset + this . position , b , offset , n ) ; this . position += n ; return n ; }


Input: Repositions the stream . [CODESPLIT] @ Override public void position ( final long newPosition ) throws IOException { flush ( ) ; if ( repositionableStream != null ) repositionableStream . position ( newPosition ) ; else if ( fileChannel != null ) fileChannel . position ( newPosition ) ; else throw new UnsupportedOperationException ( "position() can only be called if the underlying byte stream implements the RepositionableStream interface or if the getChannel() method of the underlying byte stream exists and returns a FileChannel" ) ; }


Input: This method acquires details of individual transitions that comprise a larger refresh . <p > Details of transitions in a refresh such as count and type can be useful to understand consumer performance and to troubleshoot issues relating to refresh failure . < / p > [CODESPLIT] @ Override public final void transitionsPlanned ( long beforeVersion , long desiredVersion , boolean isSnapshotPlan , List < HollowConsumer . Blob . BlobType > transitionSequence ) { updatePlanDetails . beforeVersion = beforeVersion ; updatePlanDetails . desiredVersion = desiredVersion ; updatePlanDetails . transitionSequence = transitionSequence ; if ( isSnapshotPlan ) { overallRefreshType = BlobType . SNAPSHOT ; } else { overallRefreshType = desiredVersion > beforeVersion ? BlobType . DELTA : BlobType . REVERSE_DELTA ; } refreshMetricsBuilder . setOverallRefreshType ( overallRefreshType ) ; }


Input: Metrics reporting implementation is provided by the extending subclass . If exceptions are not gracefully handled in the extending subclass then an exception there can fail the consumer refresh even though metrics reporting might not be mission critical . This method protects against that scenario by catching all exceptions logging that there was an exception and continuing with the consumer refresh . [CODESPLIT] private final void noFailRefreshEndMetricsReporting ( ConsumerRefreshMetrics refreshMetrics ) { try { refreshEndMetricsReporting ( refreshMetrics ) ; } catch ( Exception e ) { // Metric reporting is not considered critical to consumer refresh. Log exceptions and continue. log . log ( Level . SEVERE , "Encountered an exception in reporting consumer refresh metrics, ignoring exception and continuing with consumer refresh" , e ) ; } }


Input: Await successful completion of all previously submitted tasks . Throw exception of the first failed task if 1 or more tasks failed . [CODESPLIT] public void awaitSuccessfulCompletionOfCurrentTasks ( ) throws InterruptedException , ExecutionException { for ( Future < ? > f : futures ) { f . get ( ) ; } futures . clear ( ) ; }


Input: Starts the building of a { @link HashIndex } . [CODESPLIT] public static < T extends HollowRecord > Builder < T > from ( HollowConsumer consumer , Class < T > rootType ) { Objects . requireNonNull ( consumer ) ; Objects . requireNonNull ( rootType ) ; return new Builder <> ( consumer , rootType ) ; }


Input: Finds matches for a given query . [CODESPLIT] public Stream < S > findMatches ( Q query ) { Object [ ] queryArray = matchFields . stream ( ) . map ( mf -> mf . extract ( query ) ) . toArray ( ) ; HollowHashIndexResult matches = hhi . findMatches ( queryArray ) ; if ( matches == null ) { return Stream . empty ( ) ; } return matches . stream ( ) . mapToObj ( i -> selectField . extract ( api , i ) ) ; }


Input: Perform a compaction . It is expected that : [CODESPLIT] public void compact ( ) { Set < String > compactionTargets = findCompactionTargets ( ) ; Map < String , BitSet > relocatedOrdinals = new HashMap < String , BitSet > ( ) ; PartialOrdinalRemapper remapper = new PartialOrdinalRemapper ( ) ; for ( String compactionTarget : compactionTargets ) { HollowTypeReadState typeState = readEngine . getTypeState ( compactionTarget ) ; HollowTypeWriteState writeState = writeEngine . getTypeState ( compactionTarget ) ; BitSet populatedOrdinals = typeState . getListener ( PopulatedOrdinalListener . class ) . getPopulatedOrdinals ( ) ; BitSet typeRelocatedOrdinals = new BitSet ( populatedOrdinals . length ( ) ) ; int populatedCardinality = populatedOrdinals . cardinality ( ) ; writeState . addAllObjectsFromPreviousCycle ( ) ; int numRelocations = 0 ; int ordinalToRelocate = populatedOrdinals . nextSetBit ( populatedCardinality ) ; while ( ordinalToRelocate != - 1 ) { numRelocations ++ ; ordinalToRelocate = populatedOrdinals . nextSetBit ( ordinalToRelocate + 1 ) ; } HollowRecordCopier copier = HollowRecordCopier . createCopier ( typeState ) ; IntMap remappedOrdinals = new IntMap ( numRelocations ) ; ordinalToRelocate = populatedOrdinals . length ( ) ; int relocatePosition = - 1 ; try { for ( int i = 0 ; i < numRelocations ; i ++ ) { while ( ! populatedOrdinals . get ( -- ordinalToRelocate ) ) ; relocatePosition = populatedOrdinals . nextClearBit ( relocatePosition + 1 ) ; typeRelocatedOrdinals . set ( ordinalToRelocate ) ; writeState . removeOrdinalFromThisCycle ( ordinalToRelocate ) ; HollowWriteRecord rec = copier . copy ( ordinalToRelocate ) ; writeState . mapOrdinal ( rec , relocatePosition , false , true ) ; remappedOrdinals . put ( ordinalToRelocate , relocatePosition ) ; } } finally { writeState . recalculateFreeOrdinals ( ) ; } remapper . addOrdinalRemapping ( compactionTarget , remappedOrdinals ) ; relocatedOrdinals . put ( compactionTarget , typeRelocatedOrdinals ) ; } /// find the referencing dependents TransitiveSetTraverser . addReferencingOutsideClosure ( readEngine , relocatedOrdinals ) ; /// copy all forward except remapped and transitive dependents of remapped for ( HollowSchema schema : HollowSchemaSorter . dependencyOrderedSchemaList ( writeEngine . getSchemas ( ) ) ) { if ( ! compactionTargets . contains ( schema . getName ( ) ) ) { HollowTypeWriteState writeState = writeEngine . getTypeState ( schema . getName ( ) ) ; writeState . addAllObjectsFromPreviousCycle ( ) ; BitSet typeRelocatedOrdinals = relocatedOrdinals . get ( schema . getName ( ) ) ; if ( typeRelocatedOrdinals != null ) { HollowTypeReadState readState = readEngine . getTypeState ( schema . getName ( ) ) ; IntMap remappedOrdinals = new IntMap ( typeRelocatedOrdinals . cardinality ( ) ) ; boolean preserveHashPositions = shouldPreserveHashPositions ( schema ) ; HollowRecordCopier copier = HollowRecordCopier . createCopier ( readState , remapper , preserveHashPositions ) ; int remapOrdinal = typeRelocatedOrdinals . nextSetBit ( 0 ) ; while ( remapOrdinal != - 1 ) { HollowWriteRecord rec = copier . copy ( remapOrdinal ) ; int newOrdinal = writeState . add ( rec ) ; remappedOrdinals . put ( remapOrdinal , newOrdinal ) ; writeState . removeOrdinalFromThisCycle ( remapOrdinal ) ; remapOrdinal = typeRelocatedOrdinals . nextSetBit ( remapOrdinal + 1 ) ; } remapper . addOrdinalRemapping ( schema . getName ( ) , remappedOrdinals ) ; } } } }


Input: Find candidate types for compaction . No two types in the returned set will have a dependency relationship either directly or transitively . [CODESPLIT] private Set < String > findCompactionTargets ( ) { List < HollowSchema > schemas = HollowSchemaSorter . dependencyOrderedSchemaList ( readEngine . getSchemas ( ) ) ; Set < String > typesToCompact = new HashSet < String > ( ) ; for ( HollowSchema schema : schemas ) { if ( isCompactionCandidate ( schema . getName ( ) ) ) { if ( ! candidateIsDependentOnAnyTargetedType ( schema . getName ( ) , typesToCompact ) ) typesToCompact . add ( schema . getName ( ) ) ; } } return typesToCompact ; }


Input: Query an index with a single specified field . The returned value with be the ordinal of the matching record . <p > Use a generated API or the Generic Object API to use the returned ordinal . [CODESPLIT] public int getMatchingOrdinal ( Object key ) { PrimaryKeyIndexHashTable hashTable = hashTableVolatile ; if ( fieldPathIndexes . length != 1 || hashTable . bitsPerElement == 0 ) return - 1 ; int hashCode = keyHashCode ( key , 0 ) ; int ordinal = - 1 ; do { hashTable = this . hashTableVolatile ; int bucket = hashCode & hashTable . hashMask ; ordinal = readOrdinal ( hashTable , bucket ) ; while ( ordinal != - 1 ) { if ( keyDeriver . keyMatches ( key , ordinal , 0 ) ) break ; bucket ++ ; bucket &= hashTable . hashMask ; ordinal = readOrdinal ( hashTable , bucket ) ; } } while ( hashTableVolatile != hashTable ) ; return ordinal ; }


Input: Query an index with four or more specified fields . The returned value with be the ordinal of the matching record . <p > Use a generated API or the Generic Object API to use the returned ordinal . [CODESPLIT] public int getMatchingOrdinal ( Object ... keys ) { PrimaryKeyIndexHashTable hashTable = hashTableVolatile ; if ( fieldPathIndexes . length != keys . length || hashTable . bitsPerElement == 0 ) return - 1 ; int hashCode = 0 ; for ( int i = 0 ; i < keys . length ; i ++ ) hashCode ^= keyHashCode ( keys [ i ] , i ) ; int ordinal = - 1 ; do { hashTable = this . hashTableVolatile ; int bucket = hashCode & hashTable . hashMask ; ordinal = readOrdinal ( hashTable , bucket ) ; while ( ordinal != - 1 ) { if ( keyDeriver . keyMatches ( ordinal , keys ) ) break ; bucket ++ ; bucket &= hashTable . hashMask ; ordinal = readOrdinal ( hashTable , bucket ) ; } } while ( hashTableVolatile != hashTable ) ; return ordinal ; }


Input: Set the byte at the given index to the specified value [CODESPLIT] public void set ( long index , long value ) { int segmentIndex = ( int ) ( index >> log2OfSegmentSize ) ; int longInSegment = ( int ) ( index & bitmask ) ; unsafe . putOrderedLong ( segments [ segmentIndex ] , ( long ) Unsafe . ARRAY_LONG_BASE_OFFSET + ( 8 * longInSegment ) , value ) ; /// duplicate the longs here so that we can read faster. if ( longInSegment == 0 && segmentIndex != 0 ) unsafe . putOrderedLong ( segments [ segmentIndex - 1 ] , ( long ) Unsafe . ARRAY_LONG_BASE_OFFSET + ( 8 * ( 1 << log2OfSegmentSize ) ) , value ) ; }


Input: Cleans snapshot to keep the last n snapshots . Defaults to 5 . [CODESPLIT] @ Override public void cleanSnapshots ( ) { File [ ] files = getFilesByType ( HollowProducer . Blob . Type . SNAPSHOT . prefix ) ; if ( files == null || files . length <= numOfSnapshotsToKeep ) { return ; } sortByLastModified ( files ) ; for ( int i = numOfSnapshotsToKeep ; i < files . length ; i ++ ) { File file = files [ i ] ; boolean deleted = file . delete ( ) ; if ( ! deleted ) { log . warning ( "Could not delete snapshot " + file . getPath ( ) ) ; } } }


Input: Hash a field in an OBJECT record . [CODESPLIT] public static int fieldHashCode ( HollowObjectTypeDataAccess typeAccess , int ordinal , int fieldPosition ) { HollowObjectSchema schema = typeAccess . getSchema ( ) ; switch ( schema . getFieldType ( fieldPosition ) ) { case BOOLEAN : Boolean bool = typeAccess . readBoolean ( ordinal , fieldPosition ) ; return booleanHashCode ( bool ) ; case BYTES : case STRING : return typeAccess . findVarLengthFieldHashCode ( ordinal , fieldPosition ) ; case DOUBLE : double d = typeAccess . readDouble ( ordinal , fieldPosition ) ; return doubleHashCode ( d ) ; case FLOAT : float f = typeAccess . readFloat ( ordinal , fieldPosition ) ; return floatHashCode ( f ) ; case INT : return intHashCode ( typeAccess . readInt ( ordinal , fieldPosition ) ) ; case LONG : long l = typeAccess . readLong ( ordinal , fieldPosition ) ; return longHashCode ( l ) ; case REFERENCE : return typeAccess . readOrdinal ( ordinal , fieldPosition ) ; } throw new IllegalStateException ( "I don't know how to hash a " + schema . getFieldType ( fieldPosition ) ) ; }


Input: Determine whether two OBJECT field records are exactly equal . [CODESPLIT] public static boolean fieldsAreEqual ( HollowObjectTypeDataAccess typeAccess1 , int ordinal1 , int fieldPosition1 , HollowObjectTypeDataAccess typeAccess2 , int ordinal2 , int fieldPosition2 ) { HollowObjectSchema schema1 = typeAccess1 . getSchema ( ) ; switch ( schema1 . getFieldType ( fieldPosition1 ) ) { case BOOLEAN : Boolean bool1 = typeAccess1 . readBoolean ( ordinal1 , fieldPosition1 ) ; Boolean bool2 = typeAccess2 . readBoolean ( ordinal2 , fieldPosition2 ) ; return bool1 == bool2 ; case BYTES : byte [ ] data1 = typeAccess1 . readBytes ( ordinal1 , fieldPosition1 ) ; byte [ ] data2 = typeAccess2 . readBytes ( ordinal2 , fieldPosition2 ) ; return Arrays . equals ( data1 , data2 ) ; case DOUBLE : double d1 = typeAccess1 . readDouble ( ordinal1 , fieldPosition1 ) ; double d2 = typeAccess2 . readDouble ( ordinal2 , fieldPosition2 ) ; return Double . compare ( d1 , d2 ) == 0 ; case FLOAT : float f1 = typeAccess1 . readFloat ( ordinal1 , fieldPosition1 ) ; float f2 = typeAccess2 . readFloat ( ordinal2 , fieldPosition2 ) ; return Float . compare ( f1 , f2 ) == 0 ; case INT : int i1 = typeAccess1 . readInt ( ordinal1 , fieldPosition1 ) ; int i2 = typeAccess2 . readInt ( ordinal2 , fieldPosition2 ) ; return i1 == i2 ; case LONG : long l1 = typeAccess1 . readLong ( ordinal1 , fieldPosition1 ) ; long l2 = typeAccess2 . readLong ( ordinal2 , fieldPosition2 ) ; return l1 == l2 ; case STRING : String s1 = typeAccess1 . readString ( ordinal1 , fieldPosition1 ) ; return typeAccess2 . isStringFieldEqual ( ordinal2 , fieldPosition2 , s1 ) ; case REFERENCE : if ( typeAccess1 == typeAccess2 && fieldPosition1 == fieldPosition2 ) return typeAccess1 . readOrdinal ( ordinal1 , fieldPosition1 ) == typeAccess2 . readOrdinal ( ordinal2 , fieldPosition2 ) ; default : } throw new IllegalStateException ( "I don't know how to test equality for a " + schema1 . getFieldType ( fieldPosition1 ) ) ; }


Input: Augment the given selection by adding the references and the <i > transitive< / i > references of our selection . [CODESPLIT] public static void addTransitiveMatches ( HollowReadStateEngine stateEngine , Map < String , BitSet > matches ) { List < HollowSchema > schemaList = HollowSchemaSorter . dependencyOrderedSchemaList ( stateEngine ) ; Collections . reverse ( schemaList ) ; for ( HollowSchema schema : schemaList ) { BitSet currentMatches = matches . get ( schema . getName ( ) ) ; if ( currentMatches != null ) { addTransitiveMatches ( stateEngine , schema . getName ( ) , matches ) ; } } }


Input: Remove any records from the given selection which are referenced by other records not in the selection . [CODESPLIT] public static void removeReferencedOutsideClosure ( HollowReadStateEngine stateEngine , Map < String , BitSet > matches ) { List < HollowSchema > orderedSchemas = HollowSchemaSorter . dependencyOrderedSchemaList ( stateEngine ) ; Collections . reverse ( orderedSchemas ) ; for ( HollowSchema referencedSchema : orderedSchemas ) { if ( matches . containsKey ( referencedSchema . getName ( ) ) ) { for ( HollowSchema referencerSchema : orderedSchemas ) { if ( referencerSchema == referencedSchema ) break ; if ( matches . containsKey ( referencedSchema . getName ( ) ) && matches . get ( referencedSchema . getName ( ) ) . cardinality ( ) > 0 ) traverseReferencesOutsideClosure ( stateEngine , referencerSchema . getName ( ) , referencedSchema . getName ( ) , matches , REMOVE_REFERENCED_OUTSIDE_CLOSURE ) ; } } } }


Input: Augment the given selection with any records outside the selection which reference ( or transitively reference ) any records in the selection . [CODESPLIT] public static void addReferencingOutsideClosure ( HollowReadStateEngine stateEngine , Map < String , BitSet > matches ) { List < HollowSchema > orderedSchemas = HollowSchemaSorter . dependencyOrderedSchemaList ( stateEngine ) ; for ( HollowSchema referencerSchema : orderedSchemas ) { for ( HollowSchema referencedSchema : orderedSchemas ) { if ( referencedSchema == referencerSchema ) break ; if ( matches . containsKey ( referencedSchema . getName ( ) ) && matches . get ( referencedSchema . getName ( ) ) . cardinality ( ) > 0 ) traverseReferencesOutsideClosure ( stateEngine , referencerSchema . getName ( ) , referencedSchema . getName ( ) , matches , ADD_REFERENCING_OUTSIDE_CLOSURE ) ; } } }


Input: / * Note that this method is synchronized and it is the only method that modifies the { [CODESPLIT] public synchronized boolean updateTo ( long requestedVersion ) throws Throwable { if ( requestedVersion == getCurrentVersionId ( ) ) { if ( requestedVersion == HollowConstants . VERSION_NONE && hollowDataHolderVolatile == null ) { LOG . warning ( "No versions to update to, initializing to empty state" ) ; // attempting to refresh, but no available versions - initialize to empty state hollowDataHolderVolatile = newHollowDataHolder ( ) ; forceDoubleSnapshotNextUpdate ( ) ; // intentionally ignore doubleSnapshotConfig } return true ; } // Take a snapshot of the listeners to ensure additions or removals may occur concurrently // but will not take effect until a subsequent refresh final HollowConsumer . RefreshListener [ ] localListeners = refreshListeners . toArray ( new HollowConsumer . RefreshListener [ 0 ] ) ; long beforeVersion = getCurrentVersionId ( ) ; for ( HollowConsumer . RefreshListener listener : localListeners ) listener . refreshStarted ( beforeVersion , requestedVersion ) ; try { HollowUpdatePlan updatePlan = shouldCreateSnapshotPlan ( ) ? planner . planInitializingUpdate ( requestedVersion ) : planner . planUpdate ( hollowDataHolderVolatile . getCurrentVersion ( ) , requestedVersion , doubleSnapshotConfig . allowDoubleSnapshot ( ) ) ; for ( HollowConsumer . RefreshListener listener : localListeners ) if ( listener instanceof HollowConsumer . TransitionAwareRefreshListener ) ( ( HollowConsumer . TransitionAwareRefreshListener ) listener ) . transitionsPlanned ( beforeVersion , requestedVersion , updatePlan . isSnapshotPlan ( ) , updatePlan . getTransitionSequence ( ) ) ; if ( updatePlan . destinationVersion ( ) == HollowConstants . VERSION_NONE && requestedVersion != HollowConstants . VERSION_LATEST ) throw new IllegalArgumentException ( String . format ( "Could not create an update plan for version %s, because that version or any previous versions could not be retrieved." , requestedVersion ) ) ; if ( updatePlan . equals ( HollowUpdatePlan . DO_NOTHING ) && requestedVersion == HollowConstants . VERSION_LATEST ) throw new IllegalArgumentException ( "Could not create an update plan, because no existing versions could be retrieved." ) ; if ( updatePlan . destinationVersion ( requestedVersion ) == getCurrentVersionId ( ) ) return true ; if ( updatePlan . isSnapshotPlan ( ) ) { if ( hollowDataHolderVolatile == null || doubleSnapshotConfig . allowDoubleSnapshot ( ) ) { hollowDataHolderVolatile = newHollowDataHolder ( ) ; hollowDataHolderVolatile . update ( updatePlan , localListeners ) ; forceDoubleSnapshot = false ; } } else { hollowDataHolderVolatile . update ( updatePlan , localListeners ) ; } for ( HollowConsumer . RefreshListener refreshListener : localListeners ) refreshListener . refreshSuccessful ( beforeVersion , getCurrentVersionId ( ) , requestedVersion ) ; metrics . updateTypeStateMetrics ( getStateEngine ( ) , requestedVersion ) ; if ( metricsCollector != null ) metricsCollector . collect ( metrics ) ; initialLoad . complete ( getCurrentVersionId ( ) ) ; // only set the first time return getCurrentVersionId ( ) == requestedVersion ; } catch ( Throwable th ) { forceDoubleSnapshotNextUpdate ( ) ; metrics . updateRefreshFailed ( ) ; if ( metricsCollector != null ) metricsCollector . collect ( metrics ) ; for ( HollowConsumer . RefreshListener refreshListener : localListeners ) refreshListener . refreshFailed ( beforeVersion , getCurrentVersionId ( ) , requestedVersion , th ) ; // intentionally omitting a call to initialLoad.completeExceptionally(th), for producers // that write often a consumer has a chance to try another snapshot that might succeed throw th ; } }


Input: Reports metrics for when cycle is skipped due to reasons such as the producer not being the leader in a multiple - producer setting . In a multiple producer setting leader election typically favors long - lived leaders to avoid producer runs from frequently requiring to reload the full state before publishing data . When a cycle is skipped because the producer wasn t primary the current value of no . of consecutive failures and most recent cycle success time are retained and no cycle status ( success or fail ) is reported . [CODESPLIT] @ Override public void onCycleSkip ( CycleSkipReason reason ) { cycleMetricsBuilder . setConsecutiveFailures ( consecutiveFailures ) ; lastCycleSuccessTimeNanoOptional . ifPresent ( cycleMetricsBuilder :: setLastCycleSuccessTimeNano ) ; // isCycleSuccess and cycleDurationMillis are not set for skipped cycles cycleMetricsBuilder . setConsecutiveFailures ( consecutiveFailures ) ; cycleMetricsReporting ( cycleMetricsBuilder . build ( ) ) ; }


Input: Reports announcement - related metrics . [CODESPLIT] @ Override public void onAnnouncementComplete ( com . netflix . hollow . api . producer . Status status , HollowProducer . ReadState readState , long version , Duration elapsed ) { boolean isAnnouncementSuccess = false ; long dataSizeBytes = 0l ; if ( status . getType ( ) == com . netflix . hollow . api . producer . Status . StatusType . SUCCESS ) { isAnnouncementSuccess = true ; lastAnnouncementSuccessTimeNanoOptional = OptionalLong . of ( System . nanoTime ( ) ) ; } HollowReadStateEngine stateEngine = readState . getStateEngine ( ) ; dataSizeBytes = stateEngine . calcApproxDataSize ( ) ; announcementMetricsBuilder . setDataSizeBytes ( dataSizeBytes ) . setIsAnnouncementSuccess ( isAnnouncementSuccess ) . setAnnouncementDurationMillis ( elapsed . toMillis ( ) ) ; lastAnnouncementSuccessTimeNanoOptional . ifPresent ( announcementMetricsBuilder :: setLastAnnouncementSuccessTimeNano ) ; announcementMetricsReporting ( announcementMetricsBuilder . build ( ) ) ; }


Input: On cycle completion this method reports cycle metrics . [CODESPLIT] @ Override public void onCycleComplete ( com . netflix . hollow . api . producer . Status status , HollowProducer . ReadState readState , long version , Duration elapsed ) { boolean isCycleSuccess ; long cycleEndTimeNano = System . nanoTime ( ) ; if ( status . getType ( ) == com . netflix . hollow . api . producer . Status . StatusType . SUCCESS ) { isCycleSuccess = true ; consecutiveFailures = 0l ; lastCycleSuccessTimeNanoOptional = OptionalLong . of ( cycleEndTimeNano ) ; } else { isCycleSuccess = false ; consecutiveFailures ++ ; } cycleMetricsBuilder . setConsecutiveFailures ( consecutiveFailures ) . setCycleDurationMillis ( elapsed . toMillis ( ) ) . setIsCycleSuccess ( isCycleSuccess ) ; lastCycleSuccessTimeNanoOptional . ifPresent ( cycleMetricsBuilder :: setLastCycleSuccessTimeNano ) ; cycleMetricsReporting ( cycleMetricsBuilder . build ( ) ) ; }


Input: Map of string header tags reading . [CODESPLIT] private Map < String , String > readHeaderTags ( DataInputStream dis ) throws IOException { int numHeaderTags = dis . readShort ( ) ; Map < String , String > headerTags = new HashMap < String , String > ( ) ; for ( int i = 0 ; i < numHeaderTags ; i ++ ) { headerTags . put ( dis . readUTF ( ) , dis . readUTF ( ) ) ; } return headerTags ; }


Input: Adds the specified POJO to the state engine . <p > Unless previously initialized with { @link #initializeTypeState ( Class ) } the first time an instance of a particular type is added its schema is derived and added to the data model . [CODESPLIT] public int add ( Object o ) { HollowTypeMapper typeMapper = getTypeMapper ( o . getClass ( ) , null , null ) ; return typeMapper . write ( o ) ; }


Input: Warning : Experimental . the FlatRecord feature is subject to breaking changes . [CODESPLIT] @ Deprecated public void writeFlat ( Object o , FlatRecordWriter flatRecordWriter ) { HollowTypeMapper typeMapper = getTypeMapper ( o . getClass ( ) , null , null ) ; typeMapper . writeFlat ( o , flatRecordWriter ) ; }


Input: Extracts the primary key from the specified POJO . [CODESPLIT] public RecordPrimaryKey extractPrimaryKey ( Object o ) { HollowObjectTypeMapper typeMapper = ( HollowObjectTypeMapper ) getTypeMapper ( o . getClass ( ) , null , null ) ; return new RecordPrimaryKey ( typeMapper . getTypeName ( ) , typeMapper . extractPrimaryKey ( o ) ) ; }


Input: Initializes the schema for the specified type in the data model . <p > The schema will be derived from the field and type names in <code > clazz< / code > and added to the state engine s data model ; schemas of types referenced from <code > clazz< / code > will also be added . This can be used to add a type s schema to the state engine without having to add any data for that type . [CODESPLIT] public void initializeTypeState ( Class < ? > clazz ) { Objects . requireNonNull ( clazz ) ; getTypeMapper ( clazz , null , null ) ; }


Input: Clear all bits to 0 . [CODESPLIT] public void clearAll ( ) { ThreadSafeBitSetSegments segments = this . segments . get ( ) ; for ( int i = 0 ; i < segments . numSegments ( ) ; i ++ ) { AtomicLongArray segment = segments . getSegment ( i ) ; for ( int j = 0 ; j < segment . length ( ) ; j ++ ) { segment . set ( j , 0L ) ; } } }


Input: Return a new bit set which contains all bits which are contained in this bit set and which are NOT contained in the <code > other< / code > bit set . <p > [CODESPLIT] public ThreadSafeBitSet andNot ( ThreadSafeBitSet other ) { if ( other . log2SegmentSize != log2SegmentSize ) throw new IllegalArgumentException ( "Segment sizes must be the same" ) ; ThreadSafeBitSetSegments thisSegments = this . segments . get ( ) ; ThreadSafeBitSetSegments otherSegments = other . segments . get ( ) ; ThreadSafeBitSetSegments newSegments = new ThreadSafeBitSetSegments ( thisSegments . numSegments ( ) , numLongsPerSegment ) ; for ( int i = 0 ; i < thisSegments . numSegments ( ) ; i ++ ) { AtomicLongArray thisArray = thisSegments . getSegment ( i ) ; AtomicLongArray otherArray = ( i < otherSegments . numSegments ( ) ) ? otherSegments . getSegment ( i ) : null ; AtomicLongArray newArray = newSegments . getSegment ( i ) ; for ( int j = 0 ; j < thisArray . length ( ) ; j ++ ) { long thisLong = thisArray . get ( j ) ; long otherLong = ( otherArray == null ) ? 0 : otherArray . get ( j ) ; newArray . set ( j , thisLong & ~ otherLong ) ; } } ThreadSafeBitSet andNot = new ThreadSafeBitSet ( log2SegmentSize ) ; andNot . segments . set ( newSegments ) ; return andNot ; }


Input: Return a new bit set which contains all bits which are contained in * any * of the specified bit sets . [CODESPLIT] public static ThreadSafeBitSet orAll ( ThreadSafeBitSet ... bitSets ) { if ( bitSets . length == 0 ) return new ThreadSafeBitSet ( ) ; int log2SegmentSize = bitSets [ 0 ] . log2SegmentSize ; int numLongsPerSegment = bitSets [ 0 ] . numLongsPerSegment ; ThreadSafeBitSetSegments segments [ ] = new ThreadSafeBitSetSegments [ bitSets . length ] ; int maxNumSegments = 0 ; for ( int i = 0 ; i < bitSets . length ; i ++ ) { if ( bitSets [ i ] . log2SegmentSize != log2SegmentSize ) throw new IllegalArgumentException ( "Segment sizes must be the same" ) ; segments [ i ] = bitSets [ i ] . segments . get ( ) ; if ( segments [ i ] . numSegments ( ) > maxNumSegments ) maxNumSegments = segments [ i ] . numSegments ( ) ; } ThreadSafeBitSetSegments newSegments = new ThreadSafeBitSetSegments ( maxNumSegments , numLongsPerSegment ) ; AtomicLongArray segment [ ] = new AtomicLongArray [ segments . length ] ; for ( int i = 0 ; i < maxNumSegments ; i ++ ) { for ( int j = 0 ; j < segments . length ; j ++ ) { segment [ j ] = i < segments [ j ] . numSegments ( ) ? segments [ j ] . getSegment ( i ) : null ; } AtomicLongArray newSegment = newSegments . getSegment ( i ) ; for ( int j = 0 ; j < numLongsPerSegment ; j ++ ) { long value = 0 ; for ( int k = 0 ; k < segments . length ; k ++ ) { if ( segment [ k ] != null ) value |= segment [ k ] . get ( j ) ; } newSegment . set ( j , value ) ; } } ThreadSafeBitSet or = new ThreadSafeBitSet ( log2SegmentSize ) ; or . segments . set ( newSegments ) ; return or ; }


Input: Get the segment at <code > segmentIndex< / code > . If this segment does not yet exist create it . [CODESPLIT] private AtomicLongArray getSegment ( int segmentIndex ) { ThreadSafeBitSetSegments visibleSegments = segments . get ( ) ; while ( visibleSegments . numSegments ( ) <= segmentIndex ) { /// Thread safety:  newVisibleSegments contains all of the segments from the currently visible segments, plus extra. /// all of the segments in the currently visible segments are canonical and will not change. ThreadSafeBitSetSegments newVisibleSegments = new ThreadSafeBitSetSegments ( visibleSegments , segmentIndex + 1 , numLongsPerSegment ) ; /// because we are using a compareAndSet, if this thread "wins the race" and successfully sets this variable, then the segments /// which are newly defined in newVisibleSegments become canonical. if ( segments . compareAndSet ( visibleSegments , newVisibleSegments ) ) { visibleSegments = newVisibleSegments ; } else { /// If we "lose the race" and are growing the ThreadSafeBitSet segments larger, /// then we will gather the new canonical sets from the update which we missed on the next iteration of this loop. /// Newly defined segments in newVisibleSegments will be discarded, they do not get to become canonical. visibleSegments = segments . get ( ) ; } } return visibleSegments . getSegment ( segmentIndex ) ; }


Input: Updates the producer metrics : cycles completed version and type s footprint and ordinals . [CODESPLIT] public void updateCycleMetrics ( HollowProducerListener . ProducerStatus producerStatus ) { Status . StatusType st = producerStatus . getStatus ( ) == HollowProducerListener . Status . SUCCESS ? Status . StatusType . SUCCESS : Status . StatusType . FAIL ; updateCycleMetrics ( new Status ( st , producerStatus . getCause ( ) ) , producerStatus . getReadState ( ) , producerStatus . getVersion ( ) ) ; }


Input: Updates the producer metrics : cycles completed version and type s footprint and ordinals . [CODESPLIT] public void updateCycleMetrics ( Status status , HollowProducer . ReadState readState , long version ) { cyclesCompleted ++ ; if ( status . getType ( ) == Status . StatusType . FAIL ) { cycleFailed ++ ; return ; } cyclesSucceeded ++ ; if ( readState != null ) { HollowReadStateEngine hollowReadStateEngine = readState . getStateEngine ( ) ; super . update ( hollowReadStateEngine , version ) ; } else { super . update ( version ) ; } }


Input: Usage : java HollowPOJOGenerator -- argName1 = argValue1 -- argName2 == argValue2 . See { [CODESPLIT] public static void main ( String [ ] args ) throws IOException , ClassNotFoundException { if ( args . length == 0 ) { System . out . println ( "Usage:\n" + "java " + HollowPOJOGenerator . class . getName ( ) + " --arg1=value1 --arg2=value2\n" + "see " + GeneratorArguments . class . getName ( ) + " for available arguments." ) ; return ; } HollowWriteStateEngine engine = new HollowWriteStateEngine ( ) ; String packageName = null ; String pojoClassNameSuffix = null ; String pathToGeneratedFiles = null ; HollowObjectMapper mapper = new HollowObjectMapper ( engine ) ; ArgumentParser < GeneratorArguments > argumentParser = new ArgumentParser ( GeneratorArguments . class , args ) ; for ( ArgumentParser < GeneratorArguments > . ParsedArgument arg : argumentParser . getParsedArguments ( ) ) { switch ( arg . getKey ( ) ) { case addToDataModel : mapper . initializeTypeState ( HollowPOJOGenerator . class . getClassLoader ( ) . loadClass ( arg . getValue ( ) ) ) ; break ; case addSchemaFileToDataModel : HollowWriteStateCreator . readSchemaFileIntoWriteState ( arg . getValue ( ) , engine ) ; break ; case pathToGeneratedFiles : pathToGeneratedFiles = arg . getValue ( ) ; break ; case packageName : packageName = arg . getValue ( ) ; break ; case pojoClassNameSuffix : pojoClassNameSuffix = arg . getValue ( ) ; break ; default : throw new IllegalArgumentException ( "Unhandled argument " + arg . getKey ( ) ) ; } } new HollowPOJOGenerator ( packageName , pojoClassNameSuffix , engine ) . generateFiles ( pathToGeneratedFiles ) ; }


Input: Read populated ordinals as a bit set from a stream and notify a listener for each populated ordinal . [CODESPLIT] public static void readOrdinals ( DataInputStream dis , HollowTypeStateListener [ ] listeners ) throws IOException { int numLongs = dis . readInt ( ) ; int currentOrdinal = 0 ; for ( int i = 0 ; i < numLongs ; i ++ ) { long l = dis . readLong ( ) ; notifyPopulatedOrdinals ( l , currentOrdinal , listeners ) ; currentOrdinal += 64 ; } }


Input: Check if the given value is contained in the set ( or if the given value satisfies the predicate condition . ) [CODESPLIT] public boolean get ( int i ) { SparseBitSet current ; boolean result ; do { current = sparseBitSetVolatile ; result = current . get ( i ) ; } while ( current != sparseBitSetVolatile ) ; return result ; }


Input: Estimate the total number of bits used to represent the integer set . [CODESPLIT] public long size ( ) { SparseBitSet current ; long size ; do { current = sparseBitSetVolatile ; size = current . estimateBitsUsed ( ) ; } while ( current != sparseBitSetVolatile ) ; return size ; }


Input: Initialize the state engine using a snapshot blob from the provided InputStream . <p > Apply the provided { @link HollowFilterConfig } to the state . [CODESPLIT] public void readSnapshot ( InputStream is , HollowFilterConfig filter ) throws IOException { HollowBlobHeader header = readHeader ( is , false ) ; notifyBeginUpdate ( ) ; long startTime = System . currentTimeMillis ( ) ; DataInputStream dis = new DataInputStream ( is ) ; int numStates = VarInt . readVInt ( dis ) ; Collection < String > typeNames = new TreeSet < String > ( ) ; for ( int i = 0 ; i < numStates ; i ++ ) { String typeName = readTypeStateSnapshot ( dis , header , filter ) ; typeNames . add ( typeName ) ; } stateEngine . wireTypeStatesToSchemas ( ) ; long endTime = System . currentTimeMillis ( ) ; log . info ( "SNAPSHOT COMPLETED IN " + ( endTime - startTime ) + "ms" ) ; log . info ( "TYPES: " + typeNames ) ; notifyEndUpdate ( ) ; stateEngine . afterInitialization ( ) ; }


Input: Update the state engine using a delta ( or reverse delta ) blob from the provided InputStream . <p > If a { @link HollowFilterConfig } was applied at the time the { @link HollowReadStateEngine } was initialized with a snapshot it will continue to be in effect after the state is updated . [CODESPLIT] public void applyDelta ( InputStream is ) throws IOException { HollowBlobHeader header = readHeader ( is , true ) ; notifyBeginUpdate ( ) ; long startTime = System . currentTimeMillis ( ) ; DataInputStream dis = new DataInputStream ( is ) ; int numStates = VarInt . readVInt ( dis ) ; Collection < String > typeNames = new TreeSet < String > ( ) ; for ( int i = 0 ; i < numStates ; i ++ ) { String typeName = readTypeStateDelta ( dis , header ) ; typeNames . add ( typeName ) ; stateEngine . getMemoryRecycler ( ) . swap ( ) ; } long endTime = System . currentTimeMillis ( ) ; log . info ( "DELTA COMPLETED IN " + ( endTime - startTime ) + "ms" ) ; log . info ( "TYPES: " + typeNames ) ; notifyEndUpdate ( ) ; }


Input: Returns a String representation of the provided row s field value . If useFrom is true this will use the from value from the pair otherwise this will use the to value . [CODESPLIT] private static String getFieldValue ( HollowDiffViewRow row , boolean useFrom ) { Field field = useFrom ? row . getFieldPair ( ) . getFrom ( ) : row . getFieldPair ( ) . getTo ( ) ; if ( row . getFieldPair ( ) . isLeafNode ( ) ) { return field . getValue ( ) == null ? "null" : field . getValue ( ) . toString ( ) . replace ( "|" , "&#x2502" ) ; } else { String suffix = field . getValue ( ) == null ? " [null]" : "" ; return "(" + field . getTypeName ( ) + ")" + suffix ; } }


Input: Set the byte at the given index to the specified value [CODESPLIT] public void set ( long index , byte value ) { int segmentIndex = ( int ) ( index >> log2OfSegmentSize ) ; ensureCapacity ( segmentIndex ) ; segments [ segmentIndex ] [ ( int ) ( index & bitmask ) ] = value ; }


Input: Copy bytes from another ByteData to this array . [CODESPLIT] public void copy ( ByteData src , long srcPos , long destPos , long length ) { for ( long i = 0 ; i < length ; i ++ ) { set ( destPos ++ , src . get ( srcPos ++ ) ) ; } }


Input: copies exactly data . length bytes from this SegmentedByteArray into the provided byte array [CODESPLIT] public int copy ( long srcPos , byte [ ] data , int destPos , int length ) { int segmentSize = 1 << log2OfSegmentSize ; int remainingBytesInSegment = ( int ) ( segmentSize - ( srcPos & bitmask ) ) ; int dataPosition = destPos ; while ( length > 0 ) { byte [ ] segment = segments [ ( int ) ( srcPos >>> log2OfSegmentSize ) ] ; int bytesToCopyFromSegment = Math . min ( remainingBytesInSegment , length ) ; System . arraycopy ( segment , ( int ) ( srcPos & bitmask ) , data , dataPosition , bytesToCopyFromSegment ) ; dataPosition += bytesToCopyFromSegment ; srcPos += bytesToCopyFromSegment ; remainingBytesInSegment = segmentSize - ( int ) ( srcPos & bitmask ) ; length -= bytesToCopyFromSegment ; } return dataPosition - destPos ; }


Input: checks equality for a specified range of bytes in two arrays [CODESPLIT] public boolean rangeEquals ( long rangeStart , SegmentedByteArray compareTo , long cmpStart , int length ) { for ( int i = 0 ; i < length ; i ++ ) if ( get ( rangeStart + i ) != compareTo . get ( cmpStart + i ) ) return false ; return true ; }


Input: Copies the data from the provided source array into this array guaranteeing that if the update is seen by another thread then all other writes prior to this call are also visible to that thread . [CODESPLIT] public void orderedCopy ( SegmentedByteArray src , long srcPos , long destPos , long length ) { int segmentLength = 1 << log2OfSegmentSize ; int currentSegment = ( int ) ( destPos >>> log2OfSegmentSize ) ; int segmentStartPos = ( int ) ( destPos & bitmask ) ; int remainingBytesInSegment = segmentLength - segmentStartPos ; while ( length > 0 ) { int bytesToCopyFromSegment = ( int ) Math . min ( remainingBytesInSegment , length ) ; ensureCapacity ( currentSegment ) ; int copiedBytes = src . orderedCopy ( srcPos , segments [ currentSegment ] , segmentStartPos , bytesToCopyFromSegment ) ; srcPos += copiedBytes ; length -= copiedBytes ; segmentStartPos = 0 ; remainingBytesInSegment = segmentLength ; currentSegment ++ ; } }


Input: copies exactly data . length bytes from this SegmentedByteArray into the provided byte array guaranteeing that if the update is seen by another thread then all other writes prior to this call are also visible to that thread . [CODESPLIT] public int orderedCopy ( long srcPos , byte [ ] data , int destPos , int length ) { int segmentSize = 1 << log2OfSegmentSize ; int remainingBytesInSegment = ( int ) ( segmentSize - ( srcPos & bitmask ) ) ; int dataPosition = destPos ; while ( length > 0 ) { byte [ ] segment = segments [ ( int ) ( srcPos >>> log2OfSegmentSize ) ] ; int bytesToCopyFromSegment = Math . min ( remainingBytesInSegment , length ) ; orderedCopy ( segment , ( int ) ( srcPos & bitmask ) , data , dataPosition , bytesToCopyFromSegment ) ; dataPosition += bytesToCopyFromSegment ; srcPos += bytesToCopyFromSegment ; remainingBytesInSegment = segmentSize - ( int ) ( srcPos & bitmask ) ; length -= bytesToCopyFromSegment ; } return dataPosition - destPos ; }


Input: Copy bytes from the supplied InputStream into this array . [CODESPLIT] public void readFrom ( InputStream is , long length ) throws IOException { int segmentSize = 1 << log2OfSegmentSize ; int segment = 0 ; byte scratch [ ] = new byte [ segmentSize ] ; while ( length > 0 ) { ensureCapacity ( segment ) ; long bytesToCopy = Math . min ( segmentSize , length ) ; long bytesCopied = 0 ; while ( bytesCopied < bytesToCopy ) { bytesCopied += is . read ( scratch , ( int ) bytesCopied , ( int ) ( bytesToCopy - bytesCopied ) ) ; } orderedCopy ( scratch , 0 , segments [ segment ++ ] , 0 , ( int ) bytesCopied ) ; length -= bytesCopied ; } }


Input: Write a portion of this data to an OutputStream . [CODESPLIT] public void writeTo ( OutputStream os , long startPosition , long len ) throws IOException { int segmentSize = 1 << log2OfSegmentSize ; int remainingBytesInSegment = segmentSize - ( int ) ( startPosition & bitmask ) ; long remainingBytesInCopy = len ; while ( remainingBytesInCopy > 0 ) { long bytesToCopyFromSegment = Math . min ( remainingBytesInSegment , remainingBytesInCopy ) ; os . write ( segments [ ( int ) ( startPosition >>> log2OfSegmentSize ) ] , ( int ) ( startPosition & bitmask ) , ( int ) bytesToCopyFromSegment ) ; startPosition += bytesToCopyFromSegment ; remainingBytesInSegment = segmentSize - ( int ) ( startPosition & bitmask ) ; remainingBytesInCopy -= bytesToCopyFromSegment ; } }


Input: Ensures that the segment at segmentIndex exists [CODESPLIT] private void ensureCapacity ( int segmentIndex ) { while ( segmentIndex >= segments . length ) { segments = Arrays . copyOf ( segments , segments . length * 3 / 2 ) ; } if ( segments [ segmentIndex ] == null ) { segments [ segmentIndex ] = memoryRecycler . getByteArray ( ) ; } }


Input: Returns the position of a field previously added to the map or - 1 if the field has not been added to the map . [CODESPLIT] public int getPosition ( String fieldName ) { Integer index = nameFieldIndexLookup . get ( fieldName ) ; if ( index == null ) { return - 1 ; } return index ; }


Input: Called after initial pass . Returns the sum total number of select buckets in the low 7 bytes and the bits required for the max set size in the high 1 byte . [CODESPLIT] private long calculateDedupedSizesAndTotalNumberOfSelectBuckets ( MultiLinkedElementArray elementArray , GrowingSegmentedLongArray matchIndexHashAndSizeArray ) { long totalBuckets = 0 ; long maxSize = 0 ; int [ ] selectArray = new int [ 8 ] ; for ( int i = 0 ; i < elementArray . numLists ( ) ; i ++ ) { int listSize = elementArray . listSize ( i ) ; int setSize = 0 ; int predictedBuckets = HashCodes . hashTableSize ( listSize ) ; int hashMask = predictedBuckets - 1 ; if ( predictedBuckets > selectArray . length ) selectArray = new int [ predictedBuckets ] ; for ( int j = 0 ; j < predictedBuckets ; j ++ ) selectArray [ j ] = - 1 ; HollowOrdinalIterator iter = elementArray . iterator ( i ) ; int selectOrdinal = iter . next ( ) ; while ( selectOrdinal != HollowOrdinalIterator . NO_MORE_ORDINALS ) { int hash = HashCodes . hashInt ( selectOrdinal ) ; int bucket = hash & hashMask ; while ( true ) { if ( selectArray [ bucket ] == selectOrdinal ) break ; if ( selectArray [ bucket ] == - 1 ) { selectArray [ bucket ] = selectOrdinal ; setSize ++ ; break ; } bucket = ( bucket + 1 ) & hashMask ; } selectOrdinal = iter . next ( ) ; } long matchIndexHashAndSize = matchIndexHashAndSizeArray . get ( i ) ; matchIndexHashAndSize |= ( long ) setSize << 32 ; matchIndexHashAndSizeArray . set ( i , matchIndexHashAndSize ) ; totalBuckets += HashCodes . hashTableSize ( setSize ) ; if ( setSize > maxSize ) maxSize = setSize ; } return totalBuckets | ( long ) bitsRequiredToRepresentValue ( maxSize ) << 56 ; }


Input: Usage : java HollowAPIGenerator -- argName1 = argValue1 -- argName2 == argValue2 . See { [CODESPLIT] public static void main ( String [ ] args ) throws IOException , ClassNotFoundException { if ( args . length == 0 ) { System . out . println ( "Usage:\n" + "java " + HollowAPIGenerator . class . getName ( ) + " --arg1=value1 --arg2=value2\n" + "see " + GeneratorArguments . class . getName ( ) + " for available arguments." ) ; return ; } HollowWriteStateEngine engine = new HollowWriteStateEngine ( ) ; HollowAPIGenerator . Builder builder = new HollowAPIGenerator . Builder ( ) ; HollowObjectMapper mapper = new HollowObjectMapper ( engine ) ; ArgumentParser < GeneratorArguments > argumentParser = new ArgumentParser ( GeneratorArguments . class , args ) ; for ( ArgumentParser < GeneratorArguments > . ParsedArgument arg : argumentParser . getParsedArguments ( ) ) { switch ( arg . getKey ( ) ) { case addToDataModel : mapper . initializeTypeState ( HollowAPIGenerator . class . getClassLoader ( ) . loadClass ( arg . getValue ( ) ) ) ; break ; case addSchemaFileToDataModel : HollowWriteStateCreator . readSchemaFileIntoWriteState ( arg . getValue ( ) , engine ) ; break ; case apiClassName : builder . withAPIClassname ( arg . getValue ( ) ) ; break ; case classPostfix : builder . withClassPostfix ( arg . getValue ( ) ) ; break ; case getterPrefix : builder . withGetterPrefix ( arg . getValue ( ) ) ; break ; case packageName : builder . withPackageName ( arg . getValue ( ) ) ; break ; case pathToGeneratedFiles : builder . withDestination ( arg . getValue ( ) ) ; break ; case parameterizeAllClassNames : builder . withParameterizeAllClassNames ( Boolean . valueOf ( arg . getValue ( ) ) ) ; break ; default : throw new IllegalArgumentException ( "Unhandled argument " + arg . getKey ( ) ) ; } } builder . withDataModel ( engine ) . build ( ) . generateSourceFiles ( ) ; }


Input: Determines whether DataSet contains any collections schema [CODESPLIT] protected static boolean hasCollectionsInDataSet ( HollowDataset dataset ) { for ( HollowSchema schema : dataset . getSchemas ( ) ) { if ( ( schema instanceof HollowListSchema ) || ( schema instanceof HollowSetSchema ) || ( schema instanceof HollowMapSchema ) ) { return true ; } } return false ; }


Input: Generate files under the specified directory [CODESPLIT] public void generateFiles ( File directory ) throws IOException { if ( packageName != null && ! packageName . trim ( ) . isEmpty ( ) ) { String packageDir = packageName . replace ( "." , File . separator ) ; if ( ! directory . getAbsolutePath ( ) . endsWith ( packageDir ) ) { directory = new File ( directory , packageDir ) ; } } directory . mkdirs ( ) ; HollowAPIClassJavaGenerator apiClassGenerator = new HollowAPIClassJavaGenerator ( packageName , apiClassname , dataset , parameterizeClassNames , config ) ; HollowAPIFactoryJavaGenerator apiFactoryGenerator = new HollowAPIFactoryJavaGenerator ( packageName , apiClassname , dataset , config ) ; HollowHashIndexGenerator hashIndexGenerator = new HollowHashIndexGenerator ( packageName , apiClassname , dataset , config ) ; generateFile ( directory , apiClassGenerator ) ; generateFile ( directory , apiFactoryGenerator ) ; generateFile ( directory , hashIndexGenerator ) ; generateFilesForHollowSchemas ( directory ) ; }


Input: Generate files based on dataset schemas under the specified directory [CODESPLIT] protected void generateFilesForHollowSchemas ( File directory ) throws IOException { for ( HollowSchema schema : dataset . getSchemas ( ) ) { String type = schema . getName ( ) ; if ( config . isUseHollowPrimitiveTypes ( ) && HollowCodeGenerationUtils . isPrimitiveType ( type ) ) continue ; // skip if using hollow primitive type generateFile ( directory , getStaticAPIGenerator ( schema ) ) ; generateFile ( directory , getHollowObjectGenerator ( schema ) ) ; generateFile ( directory , getHollowFactoryGenerator ( schema ) ) ; if ( schema . getSchemaType ( ) == SchemaType . OBJECT ) { HollowObjectSchema objSchema = ( HollowObjectSchema ) schema ; generateFile ( directory , new HollowObjectDelegateInterfaceGenerator ( packageName , objSchema , ergonomicShortcuts , dataset , config ) ) ; generateFile ( directory , new HollowObjectDelegateCachedImplGenerator ( packageName , objSchema , ergonomicShortcuts , dataset , config ) ) ; generateFile ( directory , new HollowObjectDelegateLookupImplGenerator ( packageName , objSchema , ergonomicShortcuts , dataset , config ) ) ; generateFile ( directory , new HollowDataAccessorGenerator ( packageName , apiClassname , objSchema , dataset , config ) ) ; if ( ! config . isReservePrimaryKeyIndexForTypeWithPrimaryKey ( ) ) { generateFile ( directory , new LegacyHollowPrimaryKeyIndexGenerator ( packageName , apiClassname , objSchema , dataset , config ) ) ; } else if ( ( objSchema ) . getPrimaryKey ( ) != null ) { generateFile ( directory , new HollowPrimaryKeyIndexGenerator ( dataset , packageName , apiClassname , objSchema , config ) ) ; generateFile ( directory , new HollowUniqueKeyIndexGenerator ( packageName , apiClassname , objSchema , dataset , config ) ) ; } } } }


Input: Determine whether or not the specified ordinal contains the provided primary key value . [CODESPLIT] public boolean keyMatches ( int ordinal , Object ... keys ) { if ( keys . length != fieldPathIndexes . length ) return false ; for ( int i = 0 ; i < keys . length ; i ++ ) { if ( ! keyMatches ( keys [ i ] , ordinal , i ) ) return false ; } return true ; }


Input: Retrieve the primary key value for the specified ordinal . [CODESPLIT] public Object [ ] getRecordKey ( int ordinal ) { Object [ ] results = new Object [ fieldPathIndexes . length ] ; for ( int i = 0 ; i < fieldPathIndexes . length ; i ++ ) { results [ i ] = readValue ( ordinal , i ) ; } return results ; }


Input: Initializes the data model for the given classes . <p > Data model initialization is required prior to { @link #restore ( long HollowConsumer . BlobRetriever ) restoring } the producer . This ensures that restoration can correctly compare the producer s current data model with the data model of the restored data state and manage any differences in those models ( such as not restoring state for any types in the restoring data model not present in the producer s current data model ) . <p > After initialization a data model initialization event will be emitted to all registered data model initialization listeners { @link com . netflix . hollow . api . producer . listener . DataModelInitializationListener listeners } . [CODESPLIT] public void initializeDataModel ( Class < ? > ... classes ) { Objects . requireNonNull ( classes ) ; if ( classes . length == 0 ) { throw new IllegalArgumentException ( "classes is empty" ) ; } long start = currentTimeMillis ( ) ; for ( Class < ? > c : classes ) { objectMapper . initializeTypeState ( c ) ; } listeners . listeners ( ) . fireProducerInit ( currentTimeMillis ( ) - start ) ; isInitialized = true ; }


Input: Initializes the producer data model for the given schemas . <p > Data model initialization is required prior to { @link #restore ( long HollowConsumer . BlobRetriever ) restoring } the producer . This ensures that restoration can correctly compare the producer s current data model with the data model of the restored data state and manage any differences in those models ( such as not restoring state for any types in the restoring data model not present in the producer s current data model ) . <p > After initialization a data model initialization event will be emitted to all registered data model initialization listeners { @link com . netflix . hollow . api . producer . listener . DataModelInitializationListener listeners } . [CODESPLIT] public void initializeDataModel ( HollowSchema ... schemas ) { Objects . requireNonNull ( schemas ) ; if ( schemas . length == 0 ) { throw new IllegalArgumentException ( "classes is empty" ) ; } long start = currentTimeMillis ( ) ; HollowWriteStateCreator . populateStateEngineWithTypeWriteStates ( getWriteEngine ( ) , Arrays . asList ( schemas ) ) ; listeners . listeners ( ) . fireProducerInit ( currentTimeMillis ( ) - start ) ; isInitialized = true ; }


Input: Restores the data state to a desired version . <p > Data model { @link #initializeDataModel ( Class [] ) initialization } is required prior to restoring the producer . This ensures that restoration can correctly compare the producer s current data model with the data model of the restored data state and manage any differences in those models ( such as not restoring state for any types in the restoring data model not present in the producer s current data model ) [CODESPLIT] public HollowProducer . ReadState restore ( long versionDesired , HollowConsumer . BlobRetriever blobRetriever ) { return restore ( versionDesired , blobRetriever , ( restoreFrom , restoreTo ) -> restoreTo . restoreFrom ( restoreFrom ) ) ; }


Input: Invoke this method to alter runCycle behavior . If this Producer is not primary runCycle is a no - op . Note that by default SingleProducerEnforcer is instantiated as BasicSingleProducerEnforcer which is initialized to return true for isPrimary () [CODESPLIT] public boolean enablePrimaryProducer ( boolean doEnable ) { if ( doEnable ) { singleProducerEnforcer . enable ( ) ; } else { singleProducerEnforcer . disable ( ) ; } return ( singleProducerEnforcer . isPrimary ( ) == doEnable ) ; }


Input: / * Publish the write state storing the artifacts in the provided object . Visible for testing . [CODESPLIT] void publish ( ListenerSupport . Listeners listeners , long toVersion , Artifacts artifacts ) throws IOException { Status . StageBuilder psb = listeners . firePublishStart ( toVersion ) ; try { artifacts . snapshot = stageBlob ( listeners , blobStager . openSnapshot ( toVersion ) ) ; if ( readStates . hasCurrent ( ) ) { artifacts . delta = stageBlob ( listeners , blobStager . openDelta ( readStates . current ( ) . getVersion ( ) , toVersion ) ) ; artifacts . reverseDelta = stageBlob ( listeners , blobStager . openReverseDelta ( toVersion , readStates . current ( ) . getVersion ( ) ) ) ; publishBlob ( listeners , artifacts . delta ) ; publishBlob ( listeners , artifacts . reverseDelta ) ; if ( -- numStatesUntilNextSnapshot < 0 ) { if ( snapshotPublishExecutor == null ) { publishBlob ( listeners , artifacts . snapshot ) ; artifacts . markSnapshotPublishComplete ( ) ; } else { // Submit the publish blob task to the executor publishSnapshotBlobAsync ( listeners , artifacts ) ; } numStatesUntilNextSnapshot = numStatesBetweenSnapshots ; } else { artifacts . markSnapshotPublishComplete ( ) ; } } else { publishBlob ( listeners , artifacts . snapshot ) ; artifacts . markSnapshotPublishComplete ( ) ; numStatesUntilNextSnapshot = numStatesBetweenSnapshots ; } psb . success ( ) ; } catch ( Throwable throwable ) { psb . fail ( throwable ) ; throw throwable ; } finally { listeners . firePublishComplete ( psb ) ; } }


Input: Given these read states <p > * S ( cur ) at the currently announced version * S ( pnd ) at the pending version <p > Ensure that : <p > S ( cur ) . apply ( forwardDelta ) . checksum == S ( pnd ) . checksum S ( pnd ) . apply ( reverseDelta ) . checksum == S ( cur ) . checksum [CODESPLIT] private ReadStateHelper checkIntegrity ( ListenerSupport . Listeners listeners , ReadStateHelper readStates , Artifacts artifacts ) throws Exception { Status . StageWithStateBuilder status = listeners . fireIntegrityCheckStart ( readStates . pending ( ) ) ; try { ReadStateHelper result = readStates ; HollowReadStateEngine pending = readStates . pending ( ) . getStateEngine ( ) ; readSnapshot ( artifacts . snapshot , pending ) ; if ( readStates . hasCurrent ( ) ) { HollowReadStateEngine current = readStates . current ( ) . getStateEngine ( ) ; log . info ( "CHECKSUMS" ) ; HollowChecksum currentChecksum = HollowChecksum . forStateEngineWithCommonSchemas ( current , pending ) ; log . info ( "  CUR        " + currentChecksum ) ; HollowChecksum pendingChecksum = HollowChecksum . forStateEngineWithCommonSchemas ( pending , current ) ; log . info ( "         PND " + pendingChecksum ) ; if ( artifacts . hasDelta ( ) ) { if ( ! artifacts . hasReverseDelta ( ) ) { throw new IllegalStateException ( "Both a delta and reverse delta are required" ) ; } // FIXME: timt: future cycles will fail unless both deltas validate applyDelta ( artifacts . delta , current ) ; HollowChecksum forwardChecksum = HollowChecksum . forStateEngineWithCommonSchemas ( current , pending ) ; //out.format("  CUR => PND %s\n", forwardChecksum); if ( ! forwardChecksum . equals ( pendingChecksum ) ) { throw new HollowProducer . ChecksumValidationException ( HollowProducer . Blob . Type . DELTA ) ; } applyDelta ( artifacts . reverseDelta , pending ) ; HollowChecksum reverseChecksum = HollowChecksum . forStateEngineWithCommonSchemas ( pending , current ) ; //out.format("  CUR <= PND %s\n", reverseChecksum); if ( ! reverseChecksum . equals ( currentChecksum ) ) { throw new HollowProducer . ChecksumValidationException ( HollowProducer . Blob . Type . REVERSE_DELTA ) ; } result = readStates . swap ( ) ; } } status . success ( ) ; return result ; } catch ( Throwable th ) { status . fail ( th ) ; throw th ; } finally { listeners . fireIntegrityCheckComplete ( status ) ; } }


Input: Add a { @link HollowTypeStateListener } to a type . [CODESPLIT] public void addTypeListener ( String typeName , HollowTypeStateListener listener ) { List < HollowTypeStateListener > list = listeners . get ( typeName ) ; if ( list == null ) { list = new ArrayList < HollowTypeStateListener > ( ) ; listeners . put ( typeName , list ) ; } list . add ( listener ) ; HollowTypeReadState typeState = typeStates . get ( typeName ) ; if ( typeState != null ) typeState . addListener ( listener ) ; }


Input: Creates an object - based field path given a data set and the field path in symbolic form conforming to paths associated with a primary key . [CODESPLIT] public static FieldPath < ObjectFieldSegment > createFieldPathForPrimaryKey ( HollowDataset dataset , String type , String path ) { boolean autoExpand = ! path . endsWith ( "!" ) ; path = autoExpand ? path : path . substring ( 0 , path . length ( ) - 1 ) ; FieldPath < FieldSegment > fp = createFieldPath ( dataset , type , path , autoExpand , false , false ) ; // Erasure trick to avoid copying when it is known the list only contains // instances of ObjectFieldSegment assert fp . segments . stream ( ) . allMatch ( o -> o instanceof ObjectFieldSegment ) ; @ SuppressWarnings ( { "unchecked" , "raw" } ) FieldPath < ObjectFieldSegment > result = ( FieldPath < ObjectFieldSegment > ) ( FieldPath ) fp ; return result ; }


Input: Creates a field path given a data set and the field path in symbolic form conforming to paths associated with a hash index . [CODESPLIT] public static FieldPath < FieldSegment > createFieldPathForHashIndex ( HollowDataset dataset , String type , String path ) { return createFieldPath ( dataset , type , path , false , false , true ) ; }


Input: Creates a field path given a data set and the field path in symbolic form conforming to paths associated with a prefix index . [CODESPLIT] public static FieldPath < FieldSegment > createFieldPathForPrefixIndex ( HollowDataset dataset , String type , String path , boolean autoExpand ) { // If autoExpand is false then requireFullPath must be true boolean requireFullPath = ! autoExpand ; return createFieldPath ( dataset , type , path , autoExpand , requireFullPath , true ) ; }


Input: Creates a field path given a data set and the field path in symbolic form . [CODESPLIT] static FieldPath < FieldSegment > createFieldPath ( HollowDataset dataset , String type , String path , boolean autoExpand , boolean requireFullPath , boolean traverseSequences ) { Objects . requireNonNull ( dataset ) ; Objects . requireNonNull ( type ) ; Objects . requireNonNull ( path ) ; String [ ] segments = path . isEmpty ( ) ? new String [ 0 ] : path . split ( "\\." ) ; List < FieldSegment > fieldSegments = new ArrayList <> ( ) ; String segmentType = type ; for ( int i = 0 ; i < segments . length ; i ++ ) { HollowSchema schema = dataset . getSchema ( segmentType ) ; // @@@ Can this only occur for anything other than the root `type`? if ( schema == null ) { throw new FieldPathException ( FieldPathException . ErrorKind . NOT_BINDABLE , dataset , type , segments , fieldSegments , null , i ) ; } String segment = segments [ i ] ; HollowSchema . SchemaType schemaType = schema . getSchemaType ( ) ; if ( schemaType == HollowSchema . SchemaType . OBJECT ) { HollowObjectSchema objectSchema = ( HollowObjectSchema ) schema ; int index = objectSchema . getPosition ( segment ) ; if ( index == - 1 ) { throw new FieldPathException ( FieldPathException . ErrorKind . NOT_FOUND , dataset , type , segments , fieldSegments , schema , i ) ; } segmentType = objectSchema . getReferencedType ( index ) ; fieldSegments . add ( new ObjectFieldSegment ( objectSchema , segment , segmentType , index ) ) ; } else if ( traverseSequences && ( schemaType == HollowSchema . SchemaType . SET || schemaType == HollowSchema . SchemaType . LIST ) ) { HollowCollectionSchema collectionSchema = ( HollowCollectionSchema ) schema ; if ( ! segment . equals ( "element" ) ) { throw new FieldPathException ( FieldPathException . ErrorKind . NOT_FOUND , dataset , type , segments , fieldSegments , schema , i ) ; } segmentType = collectionSchema . getElementType ( ) ; fieldSegments . add ( new FieldSegment ( collectionSchema , segment , segmentType ) ) ; } else if ( traverseSequences && schemaType == HollowSchema . SchemaType . MAP ) { HollowMapSchema mapSchema = ( HollowMapSchema ) schema ; if ( segment . equals ( "key" ) ) { segmentType = mapSchema . getKeyType ( ) ; } else if ( segment . equals ( "value" ) ) { segmentType = mapSchema . getValueType ( ) ; } else { throw new FieldPathException ( FieldPathException . ErrorKind . NOT_FOUND , dataset , type , segments , fieldSegments , schema , i ) ; } fieldSegments . add ( new FieldSegment ( mapSchema , segment , segmentType ) ) ; } else if ( ! traverseSequences ) { throw new FieldPathException ( FieldPathException . ErrorKind . NOT_TRAVERSABLE , dataset , type , segments , fieldSegments , schema , i ) ; } if ( i < segments . length - 1 && segmentType == null ) { throw new FieldPathException ( FieldPathException . ErrorKind . NOT_TRAVERSABLE , dataset , type , segments , fieldSegments , schema , i ) ; } } if ( autoExpand ) { while ( segmentType != null ) { HollowSchema schema = dataset . getSchema ( segmentType ) ; if ( schema . getSchemaType ( ) == HollowSchema . SchemaType . OBJECT ) { HollowObjectSchema objectSchema = ( HollowObjectSchema ) schema ; if ( objectSchema . numFields ( ) == 1 ) { segmentType = objectSchema . getReferencedType ( 0 ) ; fieldSegments . add ( new ObjectFieldSegment ( objectSchema , objectSchema . getFieldName ( 0 ) , segmentType , 0 ) ) ; } else if ( objectSchema . getPrimaryKey ( ) != null && objectSchema . getPrimaryKey ( ) . numFields ( ) == 1 ) { PrimaryKey key = objectSchema . getPrimaryKey ( ) ; FieldPath < ObjectFieldSegment > expandedFieldSegments ; try { expandedFieldSegments = createFieldPathForPrimaryKey ( dataset , key . getType ( ) , key . getFieldPaths ( ) [ 0 ] ) ; } catch ( FieldPathException cause ) { FieldPathException e = new FieldPathException ( FieldPathException . ErrorKind . NOT_EXPANDABLE , dataset , type , segments , fieldSegments , objectSchema ) ; e . initCause ( cause ) ; throw e ; } fieldSegments . addAll ( expandedFieldSegments . segments ) ; break ; } else { throw new FieldPathException ( FieldPathException . ErrorKind . NOT_EXPANDABLE , dataset , type , segments , fieldSegments , objectSchema ) ; } } else { throw new FieldPathException ( FieldPathException . ErrorKind . NOT_EXPANDABLE , dataset , type , segments , fieldSegments , schema ) ; } } } else if ( requireFullPath && segmentType != null ) { throw new FieldPathException ( FieldPathException . ErrorKind . NOT_FULL , dataset , type , segments , fieldSegments ) ; } return new FieldPath <> ( type , fieldSegments , ! autoExpand ) ; }


Input: Associating the obj with an ordinal [CODESPLIT] public void put ( Object obj , int ordinal ) { int hashCode = System . identityHashCode ( obj ) ; int segment = segment ( hashCode ) ; segments [ segment ] . put ( obj , hashCode , ordinal ) ; }


Input: Returns an update plan that if executed will update the client to a version that is either equal to or as close to but less than the desired version as possible . This plan normally contains one snapshot transition and zero or more delta transitions but if no previous versions were found then an empty plan { @code HollowUpdatePlan . DO_NOTHING } is returned . [CODESPLIT] private HollowUpdatePlan snapshotPlan ( long desiredVersion ) { HollowUpdatePlan plan = new HollowUpdatePlan ( ) ; long nearestPreviousSnapshotVersion = includeNearestSnapshot ( plan , desiredVersion ) ; // The includeNearestSnapshot function returns a snapshot version that is less than or equal to the desired version if ( nearestPreviousSnapshotVersion > desiredVersion ) return HollowUpdatePlan . DO_NOTHING ; // If the nearest snapshot version is {@code HollowConstants.VERSION_LATEST} then no past snapshots were found, so // skip the delta planning and the update plan does nothing if ( nearestPreviousSnapshotVersion == HollowConstants . VERSION_LATEST ) return HollowUpdatePlan . DO_NOTHING ; plan . appendPlan ( deltaPlan ( nearestPreviousSnapshotVersion , desiredVersion , Integer . MAX_VALUE ) ) ; return plan ; }


Input: Includes the next delta only if it will not take us * after * the desired version [CODESPLIT] private long includeNextDelta ( HollowUpdatePlan plan , long currentVersion , long desiredVersion ) { HollowConsumer . Blob transition = transitionCreator . retrieveDeltaBlob ( currentVersion ) ; if ( transition != null ) { if ( transition . getToVersion ( ) <= desiredVersion ) { plan . add ( transition ) ; } return transition . getToVersion ( ) ; } return HollowConstants . VERSION_LATEST ; }


Input: initialize field positions and field paths . [CODESPLIT] private void initialize ( ) { String lastRefType = this . fieldPath . getLastRefTypeInPath ( ) ; // get all cardinality to estimate size of array bits needed. totalWords = readStateEngine . getTypeState ( lastRefType ) . getPopulatedOrdinals ( ) . cardinality ( ) ; averageWordLen = 0 ; double avg = 0 ; HollowObjectTypeReadState objectTypeReadState = ( HollowObjectTypeReadState ) readStateEngine . getTypeState ( lastRefType ) ; BitSet keyBitSet = objectTypeReadState . getPopulatedOrdinals ( ) ; int ordinal = keyBitSet . nextSetBit ( 0 ) ; while ( ordinal != - 1 ) { avg += ( ( double ) objectTypeReadState . readString ( ordinal , 0 ) . length ( ) ) / ( ( double ) objectTypeReadState . maxOrdinal ( ) ) ; ordinal = keyBitSet . nextSetBit ( ordinal + 1 ) ; } averageWordLen = ( int ) Math . ceil ( avg ) ; HollowObjectTypeReadState valueState = ( HollowObjectTypeReadState ) readStateEngine . getTypeDataAccess ( type ) ; maxOrdinalOfType = valueState . maxOrdinal ( ) ; // initialize the prefix index. build ( ) ; }


Input: Return the key to index in prefix index . Override this method to support tokens for the key . By default keys are indexed as lower case characters . <pre > { @code String [] keys = super . getKey ( ordinal ) ; String [] tokens = keys [ 0 ] . split ( ) return tokens ; } < / pre > [CODESPLIT] protected String [ ] getKeys ( int ordinal ) { Object [ ] values = fieldPath . findValues ( ordinal ) ; String [ ] stringValues = new String [ values . length ] ; for ( int i = 0 ; i < values . length ; i ++ ) { stringValues [ i ] = ( ( String ) values [ i ] ) . toLowerCase ( ) ; } return stringValues ; }


Input: Query the index to find all the ordinals that match the given prefix . Example - <pre > { @code HollowOrdinalIterator iterator = index . findKeysWithPrefix ( a ) ; int ordinal = iterator . next () ; while ( ordinal ! = HollowOrdinalIterator . NO_MORE_ORDINAL ) { // print the result using API } } < / pre > <p > For larger data sets querying smaller prefixes will be longer than querying for prefixes that are longer . [CODESPLIT] @ SuppressWarnings ( "WeakerAccess" ) public HollowOrdinalIterator findKeysWithPrefix ( String prefix ) { TST current ; HollowOrdinalIterator it ; do { current = prefixIndexVolatile ; it = current . findKeysWithPrefix ( prefix ) ; } while ( current != this . prefixIndexVolatile ) ; return it ; }


Input: Check if the given key exists in the index . [CODESPLIT] public boolean contains ( String key ) { if ( key == null ) throw new IllegalArgumentException ( "key cannot be null" ) ; TST current ; boolean result ; do { current = prefixIndexVolatile ; result = current . contains ( key ) ; } while ( current != this . prefixIndexVolatile ) ; return result ; }


Input: protected for tests [CODESPLIT] float getChangePercent ( int latestCardinality , int previousCardinality ) { int diff = Math . abs ( latestCardinality - previousCardinality ) ; return ( 100.0f * diff ) / previousCardinality ; }


Input: Add a type plus recursively add any directly or transitively referenced types . [CODESPLIT] public void addTypeRecursive ( String type , Collection < HollowSchema > schemas ) { addTypeRecursive ( type , mapSchemas ( schemas ) ) ; }


Input: Add a type plus recursively add any directly or transitively referenced types . [CODESPLIT] public void addTypeRecursive ( String type , Map < String , HollowSchema > schemas ) { addType ( type ) ; HollowSchema schema = schemas . get ( type ) ; switch ( schema . getSchemaType ( ) ) { case OBJECT : HollowObjectSchema objSchema = ( HollowObjectSchema ) schema ; for ( int i = 0 ; i < objSchema . numFields ( ) ; i ++ ) { if ( objSchema . getFieldType ( i ) == FieldType . REFERENCE ) addTypeRecursive ( objSchema . getReferencedType ( i ) , schemas ) ; } break ; case MAP : addTypeRecursive ( ( ( HollowMapSchema ) schema ) . getKeyType ( ) , schemas ) ; addTypeRecursive ( ( ( HollowMapSchema ) schema ) . getValueType ( ) , schemas ) ; break ; case LIST : case SET : addTypeRecursive ( ( ( HollowCollectionSchema ) schema ) . getElementType ( ) , schemas ) ; break ; } }


Input: Add an individual field from an OBJECT schema . This field will be either excluded or included depending on whether this is an exclude or include filter respectively . [CODESPLIT] public void addField ( String type , String objectField ) { ObjectFilterConfig typeConfig = specifiedFieldConfigs . get ( type ) ; if ( typeConfig == null ) { typeConfig = new ObjectFilterConfig ( ) ; specifiedFieldConfigs . put ( type , typeConfig ) ; } typeConfig . addField ( objectField ) ; }


Input: Add an individual field from an OBJECT schema plus recursively add any directly or transitively referenced types . This field will be either excluded or included depending on whether this is an exclude or include filter respectively . [CODESPLIT] public void addFieldRecursive ( String type , String objectField , Collection < HollowSchema > schemas ) { addFieldRecursive ( type , objectField , mapSchemas ( schemas ) ) ; }


Input: Add an individual field from an OBJECT schema plus recursively add any directly or transitively referenced types . This field will be either excluded or included depending on whether this is an exclude or include filter respectively . [CODESPLIT] public void addFieldRecursive ( String type , String objectField , Map < String , HollowSchema > schemas ) { addField ( type , objectField ) ; HollowObjectSchema schema = ( HollowObjectSchema ) schemas . get ( type ) ; if ( schema . getFieldType ( objectField ) == FieldType . REFERENCE ) { addTypeRecursive ( schema . getReferencedType ( objectField ) , schemas ) ; } }


Input: Parse a HollowFilterConfig from the specified String . The String should contain multiple lines . The first line should be either EXCLUDE or INCLUDE . Subsequent lines should be one of the following : <ul > <li > &lt ; typeName&gt ; < / li > <li > &lt ; typeName&gt ; . &lt ; fieldName&gt ; < / li > < / ul > [CODESPLIT] public static HollowFilterConfig fromString ( String conf ) { String lines [ ] = conf . split ( "\n" ) ; HollowFilterConfig config = new HollowFilterConfig ( "EXCLUDE" . equals ( lines [ 0 ] ) ) ; for ( int i = 1 ; i < lines . length ; i ++ ) { int delimiterIdx = lines [ i ] . indexOf ( ' ' ) ; if ( delimiterIdx == - 1 ) { config . addType ( lines [ i ] ) ; } else { String type = lines [ i ] . substring ( 0 , delimiterIdx ) ; String field = lines [ i ] . substring ( delimiterIdx + 1 ) ; config . addField ( type , field ) ; } } return config ; }


Input: Add a { [CODESPLIT] public void addListener ( HollowTypeStateListener listener ) { HollowTypeStateListener [ ] newListeners = Arrays . copyOf ( stateListeners , stateListeners . length + 1 ) ; newListeners [ newListeners . length - 1 ] = listener ; stateListeners = newListeners ; }


Input: Remove a specific { [CODESPLIT] public void removeListener ( HollowTypeStateListener listener ) { if ( stateListeners . length == 0 ) return ; stateListeners = Stream . of ( stateListeners ) . filter ( l -> l != listener ) . toArray ( HollowTypeStateListener [ ] :: new ) ; }


Input: Dependency types come before dependent types [CODESPLIT] public static List < HollowSchema > dependencyOrderedSchemaList ( Collection < HollowSchema > schemas ) { DependencyIndex idx = new DependencyIndex ( ) ; Map < String , HollowSchema > schemaMap = new HashMap < String , HollowSchema > ( ) ; for ( HollowSchema schema : schemas ) { schemaMap . put ( schema . getName ( ) , schema ) ; idx . indexSchema ( schema , schemas ) ; } List < HollowSchema > orderedSchemas = new ArrayList < HollowSchema > ( ) ; while ( idx . hasMoreTypes ( ) ) orderedSchemas . add ( schemaMap . get ( idx . getNextType ( ) ) ) ; return orderedSchemas ; }


Input: Make it easier to automatically use defaults for next major version [CODESPLIT] public void initWithNextMajorVersionDefaults_V3 ( ) { usePackageGrouping = true ; useBooleanFieldErgonomics = true ; reservePrimaryKeyIndexForTypeWithPrimaryKey = true ; useHollowPrimitiveTypes = true ; restrictApiToFieldType = true ; useVerboseToString = true ; }


Input: Set the byte at the given index to the specified value [CODESPLIT] public void set ( long index , long value ) { int segmentIndex = ( int ) ( index >> log2OfSegmentSize ) ; if ( segmentIndex >= segments . length ) { int nextPowerOfTwo = 1 << ( 32 - Integer . numberOfLeadingZeros ( segmentIndex ) ) ; segments = Arrays . copyOf ( segments , nextPowerOfTwo ) ; } if ( segments [ segmentIndex ] == null ) { segments [ segmentIndex ] = memoryRecycler . getLongArray ( ) ; } int longInSegment = ( int ) ( index & bitmask ) ; segments [ segmentIndex ] [ longInSegment ] = value ; }


Input: Get the value of the byte at the specified index . [CODESPLIT] public long get ( long index ) { int segmentIndex = ( int ) ( index >> log2OfSegmentSize ) ; if ( segmentIndex >= segments . length || segments [ segmentIndex ] == null ) return 0 ; int longInSegment = ( int ) ( index & bitmask ) ; return segments [ segmentIndex ] [ longInSegment ] ; }


Input: Match any records which include a field with the provided fieldName and value . [CODESPLIT] public Map < String , BitSet > findMatchingRecords ( String fieldName , String fieldValue ) { Map < String , BitSet > matches = new HashMap < String , BitSet > ( ) ; for ( HollowTypeReadState typeState : readEngine . getTypeStates ( ) ) { augmentMatchingRecords ( typeState , fieldName , fieldValue , matches ) ; } return matches ; }


Input: Match any records of the specified type which have the specified field set to the specified value . [CODESPLIT] public Map < String , BitSet > findMatchingRecords ( String typeName , String fieldName , String fieldValue ) { Map < String , BitSet > matches = new HashMap < String , BitSet > ( ) ; HollowTypeReadState typeState = readEngine . getTypeState ( typeName ) ; if ( typeState != null ) augmentMatchingRecords ( typeState , fieldName , fieldValue , matches ) ; return matches ; }


Input: Set the paths for which we will inspect differences across the two states [CODESPLIT] public void setElementMatchPaths ( String ... paths ) { resetResults ( ) ; this . elementPaths = paths ; this . elementKeyPaths = null ; this . elementNonKeyPaths = null ; }


Input: Optionally specify paths for which we will match records within an individual type s hierarchy [CODESPLIT] public void setElementKeyPaths ( String ... paths ) { resetResults ( ) ; elementKeyPaths = new BitSet ( elementPaths . length ) ; for ( int i = 0 ; i < paths . length ; i ++ ) { int elementPathIdx = getElementPathIdx ( paths [ i ] ) ; if ( elementPathIdx == - 1 ) throw new IllegalArgumentException ( "Key path must have been specified as an element match path.  Offending path: " + paths [ i ] ) ; elementKeyPaths . set ( elementPathIdx ) ; } elementNonKeyPaths = new BitSet ( elementPaths . length ) ; elementNonKeyPaths . set ( 0 , elementPaths . length ) ; elementNonKeyPaths . andNot ( elementKeyPaths ) ; }


Input: Calculate the differences [CODESPLIT] public void calculate ( ) { resetResults ( ) ; SimultaneousExecutor executor = new SimultaneousExecutor ( getClass ( ) , "calculate" ) ; final int numThreads = executor . getCorePoolSize ( ) ; for ( int i = 0 ; i < numThreads ; i ++ ) { final int threadNumber = i ; executor . execute ( new Runnable ( ) { public void run ( ) { HollowIndexerValueTraverser fromTraverser = new HollowIndexerValueTraverser ( from , type , elementPaths ) ; HollowIndexerValueTraverser toTraverser = new HollowIndexerValueTraverser ( to , type , elementPaths ) ; int hashedResults [ ] = new int [ 16 ] ; for ( int i = threadNumber ; i < matcher . getMatchedOrdinals ( ) . size ( ) ; i += numThreads ) { long ordinalPair = matcher . getMatchedOrdinals ( ) . get ( i ) ; int fromOrdinal = ( int ) ( ordinalPair >>> 32 ) ; int toOrdinal = ( int ) ordinalPair ; fromTraverser . traverse ( fromOrdinal ) ; toTraverser . traverse ( toOrdinal ) ; if ( fromTraverser . getNumMatches ( ) * 2 > hashedResults . length ) hashedResults = new int [ hashTableSize ( fromTraverser . getNumMatches ( ) ) ] ; populateHashTable ( fromTraverser , hashedResults ) ; countMatches ( fromTraverser , toTraverser , hashedResults ) ; } for ( int i = threadNumber ; i < matcher . getExtraInFrom ( ) . size ( ) ; i += numThreads ) { fromTraverser . traverse ( matcher . getExtraInFrom ( ) . get ( i ) ) ; totalUnmatchedFromElements . addAndGet ( fromTraverser . getNumMatches ( ) ) ; } for ( int i = threadNumber ; i < matcher . getExtraInTo ( ) . size ( ) ; i += numThreads ) { toTraverser . traverse ( matcher . getExtraInTo ( ) . get ( i ) ) ; totalUnmatchedToElements . addAndGet ( toTraverser . getNumMatches ( ) ) ; } } } ) ; } try { executor . awaitSuccessfulCompletion ( ) ; } catch ( Exception ex ) { throw new RuntimeException ( ex ) ; } }


Input: Should be called exclusively from the { @link HollowDiff } -- not intended for external consumption [CODESPLIT] public void addDiff ( int fromOrdinal , int toOrdinal , int score ) { if ( isSameDiffAsLastAdd ( fromOrdinal , toOrdinal ) ) { int scoreIdx = diffPairScores . size ( ) - 1 ; diffPairScores . set ( scoreIdx , diffPairScores . get ( scoreIdx ) + score ) ; } else { diffFromOrdinals . add ( fromOrdinal ) ; diffToOrdinals . add ( toOrdinal ) ; diffPairScores . add ( score ) ; } totalDiffScore += score ; }


Input: This should be called exclusively from the { [CODESPLIT] public void addResults ( HollowFieldDiff otherFieldDiff ) { for ( int i = 0 ; i < otherFieldDiff . getNumDiffs ( ) ; i ++ ) { addDiff ( otherFieldDiff . getFromOrdinal ( i ) , otherFieldDiff . getToOrdinal ( i ) , otherFieldDiff . getPairScore ( i ) ) ; } }


Input: Comparison is based on the totalDiffScore () . [CODESPLIT] @ Override public int compareTo ( HollowFieldDiff o ) { if ( o . getTotalDiffScore ( ) > totalDiffScore ) return 1 ; else if ( o . getTotalDiffScore ( ) < totalDiffScore ) return - 1 ; return 0 ; }


Input: Triggers a refresh to the latest version specified by the { [CODESPLIT] public void triggerRefresh ( ) { refreshLock . writeLock ( ) . lock ( ) ; try { updater . updateTo ( announcementWatcher == null ? Long . MAX_VALUE : announcementWatcher . getLatestVersion ( ) ) ; } catch ( Error | RuntimeException e ) { throw e ; } catch ( Throwable t ) { throw new RuntimeException ( t ) ; } finally { refreshLock . writeLock ( ) . unlock ( ) ; } }


Input: Triggers async refresh after the specified number of milliseconds has passed . <p > Any subsequent calls for async refresh will not begin until after the specified delay has completed . [CODESPLIT] public void triggerAsyncRefreshWithDelay ( int delayMillis ) { final long targetBeginTime = System . currentTimeMillis ( ) + delayMillis ; refreshExecutor . execute ( ( ) -> { try { long delay = targetBeginTime - System . currentTimeMillis ( ) ; if ( delay > 0 ) Thread . sleep ( delay ) ; } catch ( InterruptedException e ) { // Interrupting, such as shutting down the executor pool, // cancels the trigger LOG . log ( Level . INFO , "Async refresh interrupted before trigger, refresh cancelled" , e ) ; return ; } try { triggerRefresh ( ) ; } catch ( Error | RuntimeException e ) { // Ensure exceptions are propagated to the executor LOG . log ( Level . SEVERE , "Async refresh failed" , e ) ; throw e ; } } ) ; }


Input: If a { @link HollowConsumer . AnnouncementWatcher } is not specified then this method will attempt to update to the specified version and if the specified version does not exist then to a different version as specified by functionality in the { @code BlobRetriever } . <p > Otherwise an UnsupportedOperationException will be thrown . <p > This is a blocking call . [CODESPLIT] public void triggerRefreshTo ( long version ) { if ( announcementWatcher != null ) throw new UnsupportedOperationException ( "Cannot trigger refresh to specified version when a HollowConsumer.AnnouncementWatcher is present" ) ; try { updater . updateTo ( version ) ; } catch ( Error | RuntimeException e ) { throw e ; } catch ( Throwable t ) { throw new RuntimeException ( t ) ; } }


Input: Equivalent to calling { @link #getAPI () } and casting to the specified API . [CODESPLIT] public < T extends HollowAPI > T getAPI ( Class < T > apiClass ) { return apiClass . cast ( updater . getAPI ( ) ) ; }


Input: Populate the provided { @link HollowReadStateEngine } with the dataset currently in the provided { @link HollowWriteStateEngine } [CODESPLIT] public static void roundTripSnapshot ( HollowWriteStateEngine writeEngine , HollowReadStateEngine readEngine ) throws IOException { roundTripSnapshot ( writeEngine , readEngine , null ) ; }


Input: Populate the provided { @link HollowReadStateEngine } with the dataset currently in the provided { @link HollowWriteStateEngine } . <p > Apply the provided { @link HollowFilterConfig } . [CODESPLIT] public static void roundTripSnapshot ( HollowWriteStateEngine writeEngine , HollowReadStateEngine readEngine , HollowFilterConfig filter ) throws IOException { ByteArrayOutputStream baos = new ByteArrayOutputStream ( ) ; HollowBlobWriter writer = new HollowBlobWriter ( writeEngine ) ; writer . writeSnapshot ( baos ) ; writeEngine . prepareForNextCycle ( ) ; HollowBlobReader reader = new HollowBlobReader ( readEngine ) ; InputStream is = new ByteArrayInputStream ( baos . toByteArray ( ) ) ; if ( filter == null ) reader . readSnapshot ( is ) ; else reader . readSnapshot ( is , filter ) ; }


Input: Update the provided { @link HollowReadStateEngine } with the new state currently available in the { @link HollowWriteStateEngine } . <p > It is assumed that the readEngine is currently populated with the prior state from the writeEngine . [CODESPLIT] public static void roundTripDelta ( HollowWriteStateEngine writeEngine , HollowReadStateEngine readEngine ) throws IOException { ByteArrayOutputStream baos = new ByteArrayOutputStream ( ) ; HollowBlobWriter writer = new HollowBlobWriter ( writeEngine ) ; writer . writeDelta ( baos ) ; HollowBlobReader reader = new HollowBlobReader ( readEngine ) ; reader . applyDelta ( new ByteArrayInputStream ( baos . toByteArray ( ) ) ) ; writeEngine . prepareForNextCycle ( ) ; }