Class IndexFileDeleter

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable

    final class IndexFileDeleter
    extends java.lang.Object
    implements java.io.Closeable
    This class keeps track of each SegmentInfos instance that is still "live", either because it corresponds to a segments_N file in the Directory (a "commit", i.e. a committed SegmentInfos) or because it's an in-memory SegmentInfos that a writer is actively updating but has not yet committed. This class uses simple reference counting to map the live SegmentInfos instances to individual files in the Directory.

    The same directory file may be referenced by more than one IndexCommit, i.e. more than one SegmentInfos. Therefore we count how many commits reference each file. When all the commits referencing a certain file have been deleted, the refcount for that file becomes zero, and the file is deleted.

    A separate deletion policy interface (IndexDeletionPolicy) is consulted on creation (onInit) and once per commit (onCommit), to decide when a commit should be removed.

    It is the business of the IndexDeletionPolicy to choose when to delete commit points. The actual mechanics of file deletion, retrying, etc, derived from the deletion of commit points is the business of the IndexFileDeleter.

    The current default deletion policy is KeepOnlyLastCommitDeletionPolicy, which removes all prior commits when a new commit has completed. This matches the behavior before 2.2.

    Note that you must hold the write.lock before instantiating this class. It opens segments_N file(s) directly with no retry logic.

    • Constructor Detail

      • IndexFileDeleter

        public IndexFileDeleter​(java.lang.String[] files,
                                Directory directoryOrig,
                                Directory directory,
                                IndexDeletionPolicy policy,
                                SegmentInfos segmentInfos,
                                InfoStream infoStream,
                                IndexWriter writer,
                                boolean initialIndexExists,
                                boolean isReaderInit)
                         throws java.io.IOException
        Initialize the deleter: find all previous commits in the Directory, incref the files they reference, call the policy to let it delete commits. This will remove any files not referenced by any of the commits.
        Throws:
        java.io.IOException - if there is a low-level IO error
    • Method Detail

      • locked

        private boolean locked()
      • inflateGens

        static void inflateGens​(SegmentInfos infos,
                                java.util.Collection<java.lang.String> files,
                                InfoStream infoStream)
        Set all gens beyond what we currently see in the directory, to avoid double-write in cases where the previous IndexWriter did not gracefully close/rollback (e.g. os/machine crashed or lost power).
      • isClosed

        boolean isClosed()
      • deleteCommits

        private void deleteCommits()
                            throws java.io.IOException
        Remove the CommitPoints in the commitsToDelete List by DecRef'ing all files from each SegmentInfos.
        Throws:
        java.io.IOException
      • refresh

        void refresh()
              throws java.io.IOException
        Writer calls this when it has hit an error and had to roll back, to tell us that there may now be unreferenced files in the filesystem. So we re-list the filesystem and delete such files. If segmentName is non-null, we will only delete files corresponding to that segment.
        Throws:
        java.io.IOException
      • close

        public void close()
                   throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Throws:
        java.io.IOException
      • revisitPolicy

        void revisitPolicy()
                    throws java.io.IOException
        Revisits the IndexDeletionPolicy by calling its IndexDeletionPolicy.onCommit(List) again with the known commits. This is useful in cases where a deletion policy which holds onto index commits is used. The application may know that some commits are not held by the deletion policy anymore and call IndexWriter.deleteUnusedFiles(), which will attempt to delete the unused commits again.
        Throws:
        java.io.IOException
      • checkpoint

        public void checkpoint​(SegmentInfos segmentInfos,
                               boolean isCommit)
                        throws java.io.IOException
        For definition of "check point" see IndexWriter comments: "Clarification: Check Points (and commits)".

        Writer calls this when it has made a "consistent change" to the index, meaning new files are written to the index and the in-memory SegmentInfos have been modified to point to those files.

        This may or may not be a commit (segments_N may or may not have been written).

        We simply incref the files referenced by the new SegmentInfos and decref the files we had previously seen (if any).

        If this is a commit, we also call the policy to give it a chance to remove other commits. If any commits are removed, we decref their files as well.

        Throws:
        java.io.IOException
      • incRef

        void incRef​(SegmentInfos segmentInfos,
                    boolean isCommit)
             throws java.io.IOException
        Throws:
        java.io.IOException
      • incRef

        void incRef​(java.util.Collection<java.lang.String> files)
      • incRef

        void incRef​(java.lang.String fileName)
      • decRef

        void decRef​(java.util.Collection<java.lang.String> files)
             throws java.io.IOException
        Decrefs all provided files, even on exception; throws first exception hit, if any.
        Throws:
        java.io.IOException
      • decRef

        private boolean decRef​(java.lang.String fileName)
        Returns true if the file should now be deleted.
      • decRef

        void decRef​(SegmentInfos segmentInfos)
             throws java.io.IOException
        Throws:
        java.io.IOException
      • exists

        public boolean exists​(java.lang.String fileName)
      • deleteNewFiles

        void deleteNewFiles​(java.util.Collection<java.lang.String> files)
                     throws java.io.IOException
        Deletes the specified files, but only if they are new (have not yet been incref'd).
        Throws:
        java.io.IOException
      • deleteFiles

        private void deleteFiles​(java.util.Collection<java.lang.String> names)
                          throws java.io.IOException
        Throws:
        java.io.IOException
      • deleteFile

        private void deleteFile​(java.lang.String fileName)
                         throws java.io.IOException
        Throws:
        java.io.IOException