diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 0e64ad78625..8c058440ad7 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -310,7 +310,7 @@ jobs:
uses: ./.github/workflows/build-windows.yml
with:
platform: windows-x64
- msvc-toolset-version: '14.43'
+ msvc-toolset-version: '14.44'
msvc-toolset-architecture: 'x86.x64'
configure-arguments: ${{ github.event.inputs.configure-arguments }}
make-arguments: ${{ github.event.inputs.make-arguments }}
@@ -322,7 +322,7 @@ jobs:
uses: ./.github/workflows/build-windows.yml
with:
platform: windows-aarch64
- msvc-toolset-version: '14.43'
+ msvc-toolset-version: '14.44'
msvc-toolset-architecture: 'arm64'
make-target: 'hotspot'
extra-conf-options: '--openjdk-target=aarch64-unknown-cygwin'
diff --git a/doc/hotspot-style.html b/doc/hotspot-style.html
index d4c06fbd6bd..dafd29d6f54 100644
--- a/doc/hotspot-style.html
+++ b/doc/hotspot-style.html
@@ -77,6 +77,9 @@ SFINAE
thread_local
nullptr
<atomic>
+Initializing
+variables with static storage duration
Uniform Initialization
Local Function Objects
Inheriting constructors
Attributes
+noexcept
Additional Permitted
Features
@@ -791,6 +795,33 @@ differ from what the Java compilers implement.
"conservative" memory ordering, which may differ from (may be stronger
than) sequentially consistent. There are algorithms in HotSpot that are
believed to rely on that ordering.
+Initializing
+variables with static storage duration
+Variables with static storage duration and dynamic
+initialization C++14
+3.6.2). should be avoided, unless an implementation is permitted to
+perform the initialization as a static initialization. The order in
+which dynamic initializations occur is incompletely specified.
+Initialization order problems can be difficult to deal with and lead to
+surprises.
+Variables with static storage duration and non-trivial destructors
+should be avoided. HotSpot doesn't generally try to cleanup on exit, and
+running destructors at exit can lead to problems.
+Some of the approaches used in HotSpot to avoid dynamic
+initialization include:
+
+Use the Deferred<T> class template. Add a call
+to its initialization function at an appropriate place during VM
+initialization. The underlying object is never destroyed.
For objects of class type, use a variable whose value is a
+pointer to the class, initialized to nullptr. Provide an
+initialization function that sets the variable to a dynamically
+allocated object. Add a call to that function at an appropriate place
+during VM initialization. Such objects are usually never
+destroyed.
The use of uniform initialization (n2672),
@@ -1110,6 +1141,58 @@ function name and the parameter list.
memory_order_consume.
[[deprecated]] - Not relevant in HotSpot code.
+noexcept
+Use of noexcept exception specifications (n3050) are permitted with restrictions
+described below.
+
+- Only the argument-less form of
noexcept exception
+specifications are permitted.
+- Allocation functions that may return
nullptr to
+indicate allocation failure must be declared noexcept.
+- All other uses of
noexcept exception specifications are
+forbidden.
+noexcept expressions are forbidden.- Dynamic exception specifications are forbidden.
+
+HotSpot is built with exceptions disabled, e.g. compile with
+-fno-exceptions (gcc, clang) or no /EH option
+(MSVC++). So why do we need to consider noexcept at all?
+It's because noexcept exception specifications serve two
+distinct purposes.
+The first is to allow the compiler to avoid generating code or data
+in support of exceptions being thrown by a function. But this is
+unnecessary, because exceptions are disabled.
+The second is to allow the compiler and library code to choose
+different algorithms, depending on whether some function may throw
+exceptions. This is only relevant to a certain set of functions.
+
+Some allocation functions (operator new and
+operator new[]) return nullptr to indicate
+allocation failure. If a new expression calls such an
+allocation function, it must check for and handle that possibility.
+Declaring such a function noexcept informs the compiler
+that nullptr is a possible result. If an allocation
+function is not declared noexcept then the compiler may
+elide that checking and handling for a new expression
+calling that function.
Certain Standard Library facilities (notably containers) provide
+different guarantees for some operations (and may choose different
+algorithms to implement those operations), depending on whether certain
+functions (constructors, copy/move operations, swap) are nothrow or not.
+They detect this using type traits that test whether a function is
+declared noexcept. This can have a significant performance
+impact if, for example, copying is chosen over a potentially throwing
+move. But this isn't relevant, since HotSpot forbids the use of most
+Standard Library facilities.
+HotSpot code can assume no exceptions will ever be thrown, even from
+functions not declared noexcept. So HotSpot code doesn't
+ever need to check, either with conditional exception specifications or
+with noexcept expressions.
+Dynamic exception specifications were deprecated in C++11. C++17
+removed all but throw(), with that remaining a deprecated
+equivalent to noexcept.
Additional Permitted
Features
@@ -1198,12 +1281,6 @@ Library names.
href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2179.html">n2179)
— HotSpot does not permit the use of exceptions, so this feature isn't
useful.
-Avoid non-local variables with non-constexpr initialization. In
-particular, avoid variables with types requiring non-trivial
-initialization or destruction. Initialization order problems can be
-difficult to deal with and lead to surprises, as can destruction
-ordering. HotSpot doesn't generally try to cleanup on exit, and running
-destructors at exit can also lead to problems.
Avoid most operator overloading, preferring named functions. When
operator overloading is used, ensure the semantics conform to the normal
expected behavior of the operation.
+ // The region has tag `current_tag` and the operation has tag `operation_tag`.
+ // For each state, we decide how much to be added/subtracted from current_tag to operation_tag. Two tables for reserve and commit.
+ // Each pair of in the table means add `x` to current_tag and add `y` to operation_tag. There are 3 pairs in each row for 3 states.
+ // For example, `reserve[1][4,5]` says `-a,a` means:
+ // - we are reserving with operation_tag a region which is already commited with current_tag
+ // - since we are reserving, then `a` will be added to operation_tag. (`y` is `a`)
+ // - since we uncommitting (by reserving) then `a` is to be subtracted from current_tag. (`x` is `-a`).
+ // - amount of uncommitted size is in table `commit[1][4,5]` which is `-a,0` that means subtract `a` from current_tag.
+ // existing state
+ SingleDiff::delta reserve[4][3*2] = {// Rl Rs C
+ {0,0, -a,0, -a,0 }, // op == Release
+ {0,a, -a,a, -a,a }, // op == Reserve
+ {0,a, -a,a, -a,a }, // op == Commit
+ {0,0, 0,0, 0,0 } // op == Uncommit
+ };
+ SingleDiff::delta commit[4][3*2] = {// Rl Rs C
+ {0,0, 0,0, -a,0 }, // op == Release
+ {0,0, 0,0, -a,0 }, // op == Reserve
+ {0,a, 0,a, -a,a }, // op == Commit
+ {0,0, 0,0, -a,0 } // op == Uncommit
+ };
+ SingleDiff& from_rescom = diff.tag[NMTUtil::tag_to_index(current_tag)];
+ SingleDiff& to_rescom = diff.tag[NMTUtil::tag_to_index(operation_tag)];
+ int st = state_to_index(ex);
+ from_rescom.reserve += reserve[op][st * 2 ];
+ to_rescom.reserve += reserve[op][st * 2 + 1];
+ from_rescom.commit += commit[op][st * 2 ];
+ to_rescom.commit += commit[op][st * 2 + 1];
+
+}
+// update the region state between n1 and n2. Since n1 and n2 are pointers, any update of them will be visible from tree.
+// If n1 is noop, it can be removed because its left region (n1->val().in) is already decided and its right state (n1->val().out) is decided here.
+// The state of right of n2 (n2->val().out) cannot be decided here yet.
+void VMATree::update_region(TreapNode* n1, TreapNode* n2, const RequestInfo& req, SummaryDiff& diff) {
+ assert(n1 != nullptr,"sanity");
+ assert(n2 != nullptr,"sanity");
+ //.........n1......n2......
+ // ^------^
+ // |
+ IntervalState exSt = n1->val().out; // existing state info
+
+
+ StateType existing_state = exSt.type();
+ MemTag existing_tag = exSt.mem_tag();
+ SIndex existing_reserve_callstack = exSt.reserved_stack();
+ SIndex existing_commit_callstack = exSt.committed_stack();
+
+ StateType new_state = get_new_state(existing_state, req);
+ MemTag new_tag = get_new_tag(n1->val().out.mem_tag(), req);
+ SIndex new_reserve_callstack = get_new_reserve_callstack(existing_reserve_callstack, existing_state, req);
+ SIndex new_commit_callstack = get_new_commit_callstack(existing_commit_callstack, existing_state, req);
+
+ // n1........n2
+ // out-->
+ n1->val().out.set_tag(new_tag);
+ n1->val().out.set_type(new_state);
+ n1->val().out.set_reserve_stack(new_reserve_callstack);
+ n1->val().out.set_commit_stack(new_commit_callstack);
+
+ // n1........n2
+ // <--in
+ n2->val().in.set_tag(new_tag);
+ n2->val().in.set_type(new_state);
+ n2->val().in.set_reserve_stack(new_reserve_callstack);
+ n2->val().in.set_commit_stack(new_commit_callstack);
+
+ SingleDiff::delta region_size = n2->key() - n1->key();
+ compute_summary_diff(region_size, existing_tag, existing_state, req, new_tag, diff);
+}
+
+
+VMATree::SummaryDiff VMATree::register_mapping(position _A, position _B, StateType state,
const RegionData& metadata, bool use_tag_inplace) {
- assert(!use_tag_inplace || metadata.mem_tag == mtNone,
- "If using use_tag_inplace, then the supplied tag should be mtNone, was instead: %s", NMTUtil::tag_to_name(metadata.mem_tag));
- if (A == B) {
- // A 0-sized mapping isn't worth recording.
+
+ if (_A == _B) {
return SummaryDiff();
}
-
+ assert(_A < _B, "should be");
+ SummaryDiff diff;
+ RequestInfo req{_A, _B, state, metadata.mem_tag, metadata.stack_idx, use_tag_inplace};
IntervalChange stA{
IntervalState{StateType::Released, empty_regiondata},
IntervalState{ state, metadata}
@@ -51,176 +259,401 @@ VMATree::SummaryDiff VMATree::register_mapping(position A, position B, StateType
IntervalState{ state, metadata},
IntervalState{StateType::Released, empty_regiondata}
};
+ stA.out.set_commit_stack(NativeCallStackStorage::invalid);
+ stB.in.set_commit_stack(NativeCallStackStorage::invalid);
+ VMATreap::Range rA = _tree.find_enclosing_range(_A);
+ VMATreap::Range rB = _tree.find_enclosing_range(_B);
- // First handle A.
- // Find closest node that is LEQ A
- bool LEQ_A_found = false;
- AddressState LEQ_A;
- TreapNode* leqA_n = _tree.closest_leq(A);
- if (leqA_n == nullptr) {
- assert(!use_tag_inplace, "Cannot use the tag inplace if no pre-existing tag exists. From: " PTR_FORMAT " To: " PTR_FORMAT, A, B);
- if (use_tag_inplace) {
- log_debug(nmt)("Cannot use the tag inplace if no pre-existing tag exists. From: " PTR_FORMAT " To: " PTR_FORMAT, A, B);
+ // nodes: .....X.......Y...Z......W........U
+ // request: A------------------B
+ // X,Y = enclosing_nodes(A)
+ // W,U = enclosing_nodes(B)
+ // The cases are whether or not X and Y exists and X == A. (A == Y doesn't happen since it is searched by 'lt' predicate)
+ // The cases are whether or not W and U exists and W == B. (B == U doesn't happen since it is searched by 'lt' predicate)
+
+ // We update regions in 3 sections: 1) X..A..Y, 2) Y....W, 3) W..B..U
+ // Y: is the closest node greater than A, but less than B
+ // W: is the closest node less than B, but greater than A
+ // The regions in [Y,W) are updated in a loop. We update X..A..Y before the loop and W..B..U after the loop.
+ // The table below summarizes the overlap cases. The overlapping case depends on whether X, Y, W and U exist or not,
+ // and if they exist whether they are the same or not.
+ // In the notations here, when there is not dot ('.') between two nodes it meaans that they are the same. For example,
+ // ...XA....Y.... means X == A.
+
+
+ // row 0: .........A..................B.....
+ // row 1: .........A...YW.............B..... // it is impossible, since it means only one node exists in the tree.
+ // row 2: .........A...Y..........W...B.....
+ // row 3: .........A...Y.............WB.....
+
+ // row 4: .....X...A..................B.....
+ // row 5: .....X...A...YW.............B.....
+ // row 6: .....X...A...Y..........W...B.....
+ // row 7: .....X...A...Y.............WB.....
+
+ // row 8: ........XA..................B.....
+ // row 9: ........XA...YW.............B.....
+ // row 10: ........XA...Y..........W...B.....
+ // row 11: ........XA...Y.............WB.....
+
+ // row 12: .........A..................B....U
+ // row 13: .........A...YW.............B....U
+ // row 14: .........A...Y..........W...B....U
+ // row 15: .........A...Y.............WB....U
+
+ // row 16: .....X...A..................B....U
+ // row 17: .....X...A...YW.............B....U
+ // row 18: .....X...A...Y..........W...B....U
+ // row 19: .....X...A...Y.............WB....U
+
+ // row 20: ........XA..................B....U
+ // row 21: ........XA...YW.............B....U
+ // row 22: ........XA...Y..........W...B....U
+ // row 23: ........XA...Y.............WB....U
+
+
+ // We intentionally did not summarize/compress the cases to keep them as separate.
+ // This expanded way of describing the cases helps us to understand/analyze/verify/debug/maintain
+ // the corresponding code more easily.
+ // Mapping of table to row, row to switch-case should be consistent. If one changes, the others have
+ // to be updated accordingly. The sequence of dependecies is: table -> row no -> switch(row)-case -> code.
+ // Meaning that whenever any of one item in this sequence is changed, the rest of the consequent items to
+ // be checked/changed.
+
+ TreapNode* X = rA.start;
+ TreapNode* Y = rA.end;
+ TreapNode* W = rB.start;
+ TreapNode* U = rB.end;
+ TreapNode nA{_A, stA, 0}; // the node that represents A
+ TreapNode nB{_B, stB, 0}; // the node that represents B
+ TreapNode* A = &nA;
+ TreapNode* B = &nB;
+ auto upsert_if= [&](TreapNode* node) {
+ if (!node->val().is_noop()) {
+ _tree.upsert(node->key(), node->val());
}
- // No match. We add the A node directly, unless it would have no effect.
- if (!stA.is_noop()) {
- _tree.upsert(A, stA);
+ };
+ // update region between n1 and n2
+ auto update = [&](TreapNode* n1, TreapNode* n2) {
+ update_region(n1, n2, req, diff);
+ };
+ auto remove_if = [&](TreapNode* node) -> bool{
+ if (node->val().is_noop()) {
+ _tree.remove(node->key());
+ return true;
}
- } else {
- LEQ_A_found = true;
- LEQ_A = AddressState{leqA_n->key(), leqA_n->val()};
- StateType leqA_state = leqA_n->val().out.type();
- StateType new_state = stA.out.type();
- // If we specify use_tag_inplace then the new region takes over the current tag instead of the tag in metadata.
- // This is important because the VirtualMemoryTracker API doesn't require supplying the tag for some operations.
- if (use_tag_inplace) {
- assert(leqA_n->val().out.type() != StateType::Released, "Should not use inplace the tag of a released region");
- MemTag tag = leqA_n->val().out.mem_tag();
- stA.out.set_tag(tag);
- stB.in.set_tag(tag);
- }
-
- // Unless we know better, let B's outgoing state be the outgoing state of the node at or preceding A.
- // Consider the case where the found node is the start of a region enclosing [A,B)
- stB.out = out_state(leqA_n);
-
- // Direct address match.
- if (leqA_n->key() == A) {
- // Take over in state from old address.
- stA.in = in_state(leqA_n);
-
- // We may now be able to merge two regions:
- // If the node's old state matches the new, it becomes a noop. That happens, for example,
- // when expanding a committed area: commit [x1, A); ... commit [A, x3)
- // and the result should be a larger area, [x1, x3). In that case, the middle node (A and le_n)
- // is not needed anymore. So we just remove the old node.
- stB.in = stA.out;
- if (stA.is_noop()) {
- // invalidates leqA_n
- _tree.remove(leqA_n->key());
- } else {
- // If the state is not matching then we have different operations, such as:
- // reserve [x1, A); ... commit [A, x2); or
- // reserve [x1, A), mem_tag1; ... reserve [A, x2), mem_tag2; or
- // reserve [A, x1), mem_tag1; ... reserve [A, x2), mem_tag2;
- // then we re-use the existing out node, overwriting its old metadata.
- leqA_n->val() = stA;
+ return false;
+ };
+ GrowableArrayCHeap to_be_removed;
+ // update regions in range A to B
+ auto update_loop = [&]() {
+ TreapNode* prev = nullptr;
+ _tree.visit_range_in_order(_A + 1, _B + 1, [&](TreapNode* curr) {
+ if (prev != nullptr) {
+ update_region(prev, curr, req, diff);
+ // during visit, structure of the tree should not be changed
+ // keep the keys to be removed, and remove them later
+ if (prev->val().is_noop()) {
+ to_be_removed.push(prev->key());
+ }
}
- } else {
- // The address must be smaller.
- assert(A > leqA_n->key(), "must be");
+ prev = curr;
+ return true;
+ });
+ };
+ // update region of [A,T)
+ auto update_A = [&](TreapNode* T) {
+ A->val().out = A->val().in;
+ update(A, T);
+ };
+ bool X_exists = X != nullptr;
+ bool Y_exists = Y != nullptr && Y->key() <= _B;
+ bool W_exists = W != nullptr && W->key() > _A;
+ bool U_exists = U != nullptr;
+ bool X_eq_A = X_exists && X->key() == _A;
+ bool W_eq_B = W_exists && W->key() == _B;
+ bool Y_eq_W = Y_exists && W_exists && W->key() == Y->key();
+ int row = -1;
+#ifdef ASSERT
+ auto print_case = [&]() {
+ log_trace(vmatree)(" req: %4d---%4d", (int)_A, (int)_B);
+ log_trace(vmatree)(" row: %2d", row);
+ log_trace(vmatree)(" X: %4ld", X_exists ? (long)X->key() : -1);
+ log_trace(vmatree)(" Y: %4ld", Y_exists ? (long)Y->key() : -1);
+ log_trace(vmatree)(" W: %4ld", W_exists ? (long)W->key() : -1);
+ log_trace(vmatree)(" U: %4ld", U_exists ? (long)U->key() : -1);
+ };
+#endif
+ // Order of the nodes if they exist are as: X <= A < Y <= W <= B < U
+ // A---------------------------B
+ // X Y YW WB U
+ // XA Y YW WB U
+ if (!X_exists && !Y_exists && !U_exists) { row = 0; }
+ if (!X_exists && Y_exists && Y_eq_W && !W_eq_B && !U_exists) { row = 1; }
+ if (!X_exists && Y_exists && !Y_eq_W && !W_eq_B && !U_exists) { row = 2; }
+ if (!X_exists && Y_exists && W_eq_B && !U_exists) { row = 3; }
- // We add a new node, but only if there would be a state change. If there would not be a
- // state change, we just omit the node.
- // That happens, for example, when reserving within an already reserved region with identical metadata.
- stA.in = out_state(leqA_n); // .. and the region's prior state is the incoming state
- if (stA.is_noop()) {
- // Nothing to do.
- } else {
- // Add new node.
- _tree.upsert(A, stA);
- }
+ if ( X_exists && !Y_exists && !U_exists) { row = 4; }
+ if ( X_exists && Y_exists && Y_eq_W && !W_eq_B && !U_exists) { row = 5; }
+ if ( X_exists && Y_exists && !Y_eq_W && !W_eq_B && !U_exists) { row = 6; }
+ if ( X_exists && Y_exists && W_eq_B && !U_exists) { row = 7; }
+
+ if ( X_eq_A && !Y_exists && !U_exists) { row = 8; }
+ if ( X_eq_A && Y_exists && Y_eq_W && !W_eq_B && !U_exists) { row = 9; }
+ if ( X_eq_A && Y_exists && !Y_eq_W && !W_eq_B && !U_exists) { row = 10; }
+ if ( X_eq_A && Y_exists && W_eq_B && !U_exists) { row = 11; }
+
+ if (!X_exists && !Y_exists && U_exists) { row = 12; }
+ if (!X_exists && Y_exists && Y_eq_W && !W_eq_B && U_exists) { row = 13; }
+ if (!X_exists && Y_exists && !Y_eq_W && !W_eq_B && U_exists) { row = 14; }
+ if (!X_exists && Y_exists && W_eq_B && U_exists) { row = 15; }
+
+ if ( X_exists && !Y_exists && U_exists) { row = 16; }
+ if ( X_exists && Y_exists && Y_eq_W && !W_eq_B && U_exists) { row = 17; }
+ if ( X_exists && Y_exists && !Y_eq_W && !W_eq_B && U_exists) { row = 18; }
+ if ( X_exists && Y_exists && W_eq_B && U_exists) { row = 19; }
+
+ if ( X_eq_A && !Y_exists && U_exists) { row = 20; }
+ if ( X_eq_A && Y_exists && Y_eq_W && !W_eq_B && U_exists) { row = 21; }
+ if ( X_eq_A && Y_exists && !Y_eq_W && !W_eq_B && U_exists) { row = 22; }
+ if ( X_eq_A && Y_exists && W_eq_B && U_exists) { row = 23; }
+
+ switch(row) {
+ // row 0: .........A..................B.....
+ case 0: {
+ update_A(B);
+ upsert_if(A);
+ upsert_if(B);
+ break;
}
- }
-
- // Now we handle B.
- // We first search all nodes that are (A, B]. All of these nodes
- // need to be deleted and summary accounted for. The last node before B determines B's outgoing state.
- // If there is no node between A and B, its A's incoming state.
- GrowableArrayCHeap to_be_deleted_inbetween_a_b;
- bool B_needs_insert = true;
-
- // Find all nodes between (A, B] and record their addresses and values. Also update B's
- // outgoing state.
- _tree.visit_range_in_order(A + 1, B + 1, [&](TreapNode* head) {
- int cmp_B = PositionComparator::cmp(head->key(), B);
- stB.out = out_state(head);
- if (cmp_B < 0) {
- // Record all nodes preceding B.
- to_be_deleted_inbetween_a_b.push({head->key(), head->val()});
- } else if (cmp_B == 0) {
- // Re-purpose B node, unless it would result in a noop node, in
- // which case record old node at B for deletion and summary accounting.
- if (stB.is_noop()) {
- to_be_deleted_inbetween_a_b.push(AddressState{B, head->val()});
- } else {
- head->val() = stB;
- }
- B_needs_insert = false;
+ // row 1: .........A...YW.............B.....
+ case 1: {
+ ShouldNotReachHere();
+ break;
}
- });
-
- // Insert B node if needed
- if (B_needs_insert && // Was not already inserted
- !stB.is_noop()) // The operation is differing
- {
- _tree.upsert(B, stB);
- }
-
- // We now need to:
- // a) Delete all nodes between (A, B]. Including B in the case of a noop.
- // b) Perform summary accounting
- SummaryDiff diff;
-
- if (to_be_deleted_inbetween_a_b.length() == 0 && LEQ_A_found) {
- // We must have smashed a hole in an existing region (or replaced it entirely).
- // LEQ_A < A < B <= C
- SingleDiff& rescom = diff.tag[NMTUtil::tag_to_index(LEQ_A.out().mem_tag())];
- if (LEQ_A.out().type() == StateType::Reserved) {
- rescom.reserve -= B - A;
- } else if (LEQ_A.out().type() == StateType::Committed) {
- rescom.commit -= B - A;
- rescom.reserve -= B - A;
+ // row 2: .........A...Y..........W...B.....
+ case 2: {
+ update_A(Y);
+ upsert_if(A);
+ update_loop();
+ remove_if(Y);
+ update(W, B);
+ remove_if(W);
+ upsert_if(B);
+ break;
}
- }
-
- // Track the previous node.
- AddressState prev{A, stA};
- for (int i = 0; i < to_be_deleted_inbetween_a_b.length(); i++) {
- const AddressState delete_me = to_be_deleted_inbetween_a_b.at(i);
- _tree.remove(delete_me.address);
-
- // Perform summary accounting
- SingleDiff& rescom = diff.tag[NMTUtil::tag_to_index(delete_me.in().mem_tag())];
- if (delete_me.in().type() == StateType::Reserved) {
- rescom.reserve -= delete_me.address - prev.address;
- } else if (delete_me.in().type() == StateType::Committed) {
- rescom.commit -= delete_me.address - prev.address;
- rescom.reserve -= delete_me.address - prev.address;
+ // row 3: .........A...Y.............WB.....
+ case 3: {
+ update_A(Y);
+ upsert_if(A);
+ update_loop();
+ remove_if(W);
+ break;
}
- prev = delete_me;
- }
-
- if (prev.address != A && prev.out().type() != StateType::Released) {
- // The last node wasn't released, so it must be connected to a node outside of (A, B)
- // A - prev - B - (some node >= B)
- // It might be that prev.address == B == (some node >= B), this is fine.
- if (prev.out().type() == StateType::Reserved) {
- SingleDiff& rescom = diff.tag[NMTUtil::tag_to_index(prev.out().mem_tag())];
- rescom.reserve -= B - prev.address;
- } else if (prev.out().type() == StateType::Committed) {
- SingleDiff& rescom = diff.tag[NMTUtil::tag_to_index(prev.out().mem_tag())];
- rescom.commit -= B - prev.address;
- rescom.reserve -= B - prev.address;
+ // row 4: .....X...A..................B.....
+ case 4: {
+ A->val().in = X->val().out;
+ update_A(B);
+ upsert_if(A);
+ upsert_if(B);
+ break;
}
+ // row 5: .....X...A...YW.............B.....
+ case 5: {
+ A->val().in = X->val().out;
+ update_A(Y);
+ upsert_if(A);
+ update(Y, B);
+ remove_if(Y);
+ upsert_if(B);
+ break;
+ }
+ // row 6: .....X...A...Y..........W...B.....
+ case 6: {
+ A->val().in = X->val().out;
+ update_A(Y);
+ upsert_if(A);
+ update_loop();
+ update(W, B);
+ remove_if(W);
+ upsert_if(B);
+ break;
+ }
+ // row 7: .....X...A...Y.............WB.....
+ case 7: {
+ A->val().in = X->val().out;
+ update_A(Y);
+ upsert_if(A);
+ update_loop();
+ remove_if(W);
+ break;
+ }
+ // row 8: ........XA..................B.....
+ case 8: {
+ update(X, B);
+ remove_if(X);
+ upsert_if(B);
+ break;
+ }
+ // row 9: ........XA...YW.............B.....
+ case 9: {
+ update(X, Y);
+ remove_if(X);
+ update(W, B);
+ remove_if(W);
+ upsert_if(B);
+ break;
+ }
+ // row 10: ........XA...Y..........W...B.....
+ case 10: {
+ update(X, Y);
+ remove_if(X);
+ update_loop();
+ update(W, B);
+ remove_if(W);
+ upsert_if(B);
+ break;
+ }
+ // row 11: ........XA...Y.............WB.....
+ case 11: {
+ update(X, Y);
+ remove_if(X);
+ update_loop();
+ remove_if(W);
+ break;
+ }
+ // row 12: .........A..................B....U
+ case 12: {
+ update_A(B);
+ upsert_if(A);
+ upsert_if(B);
+ break;
+ }
+ // row 13: .........A...YW.............B....U
+ case 13: {
+ update_A(Y);
+ upsert_if(A);
+ update(W, B);
+ remove_if(W);
+ B->val().out = U->val().in;
+ upsert_if(B);
+ break;
+ }
+ // row 14: .........A...Y..........W...B....U
+ case 14: {
+ update_A(Y);
+ upsert_if(A);
+ update_loop();
+ update(W, B);
+ remove_if(W);
+ B->val().out = U->val().in;
+ upsert_if(B);
+ break;
+ }
+ // row 15: .........A...Y.............WB....U
+ case 15: {
+ update_A(Y);
+ upsert_if(A);
+ update_loop();
+ remove_if(W);
+ break;
+ }
+ // row 16: .....X...A..................B....U
+ case 16: {
+ A->val().in = X->val().out;
+ update_A(B);
+ upsert_if(A);
+ B->val().out = U->val().in;
+ upsert_if(B);
+ break;
+ }
+ // row 17: .....X...A...YW.............B....U
+ case 17: {
+ A->val().in = X->val().out;
+ update_A(Y);
+ upsert_if(A);
+ update(W, B);
+ remove_if(W);
+ B->val().out = U->val().in;
+ upsert_if(B);
+ break;
+ }
+ // row 18: .....X...A...Y..........W...B....U
+ case 18: {
+ A->val().in = X->val().out;
+ update_A(Y);
+ upsert_if(A);
+ update_loop();
+ update(W, B);
+ remove_if(W);
+ B->val().out = U->val().in;
+ upsert_if(B);
+ break;
+ }
+ // row 19: .....X...A...Y.............WB....U
+ case 19: {
+ A->val().in = X->val().out;
+ update_A(Y);
+ upsert_if(A);
+ update_loop();
+ remove_if(W);
+ break;
+ }
+ // row 20: ........XA..................B....U
+ case 20: {
+ update(X, B);
+ remove_if(X);
+ B->val().out = U->val().in;
+ upsert_if(B);
+ break;
+ }
+ // row 21: ........XA...YW.............B....U
+ case 21: {
+ update(X, Y);
+ remove_if(X);
+ update(W, B);
+ remove_if(W);
+ B->val().out = U->val().in;
+ upsert_if(B);
+ break;
+ }
+ // row 22: ........XA...Y..........W...B....U
+ case 22: {
+ update(X, Y);
+ remove_if(X);
+ update_loop();
+ update(W, B);
+ remove_if(W);
+ B->val().out = U->val().in;
+ upsert_if(B);
+ break;
+ }
+ // row 23: ........XA...Y.............WB....U
+ case 23: {
+ update(X, Y);
+ remove_if(X);
+ update_loop();
+ remove_if(W);
+ break;
+ }
+ default:
+ ShouldNotReachHere();
}
- // Finally, we can register the new region [A, B)'s summary data.
- SingleDiff& rescom = diff.tag[NMTUtil::tag_to_index(stA.out.mem_tag())];
- if (state == StateType::Reserved) {
- rescom.reserve += B - A;
- } else if (state == StateType::Committed) {
- rescom.commit += B - A;
- rescom.reserve += B - A;
+ // Remove the 'noop' nodes that found inside the loop
+ while(to_be_removed.length() != 0) {
+ _tree.remove(to_be_removed.pop());
}
+
return diff;
}
#ifdef ASSERT
void VMATree::print_on(outputStream* out) {
visit_in_order([&](TreapNode* current) {
- out->print("%zu (%s) - %s - ", current->key(), NMTUtil::tag_to_name(out_state(current).mem_tag()),
- statetype_to_string(out_state(current).type()));
+ out->print("%zu (%s) - %s [%d, %d]-> ", current->key(), NMTUtil::tag_to_name(out_state(current).mem_tag()),
+ statetype_to_string(out_state(current).type()), current->val().out.reserved_stack(), current->val().out.committed_stack());
+ return true;
});
out->cr();
}
@@ -268,7 +701,7 @@ VMATree::SummaryDiff VMATree::set_tag(const position start, const size size, con
SummaryDiff diff;
// Ignore any released ranges, these must be mtNone and have no stack
if (type != StateType::Released) {
- RegionData new_data = RegionData(out.stack(), tag);
+ RegionData new_data = RegionData(out.reserved_stack(), tag);
SummaryDiff result = register_mapping(from, end, type, new_data);
diff.add(result);
}
@@ -289,7 +722,7 @@ VMATree::SummaryDiff VMATree::set_tag(const position start, const size size, con
StateType type = out.type();
if (type != StateType::Released) {
- RegionData new_data = RegionData(out.stack(), tag);
+ RegionData new_data = RegionData(out.reserved_stack(), tag);
SummaryDiff result = register_mapping(from, end, type, new_data);
diff.add(result);
}
diff --git a/src/hotspot/share/nmt/vmatree.hpp b/src/hotspot/share/nmt/vmatree.hpp
index 0c639e929b7..7f52b406309 100644
--- a/src/hotspot/share/nmt/vmatree.hpp
+++ b/src/hotspot/share/nmt/vmatree.hpp
@@ -26,6 +26,7 @@
#ifndef SHARE_NMT_VMATREE_HPP
#define SHARE_NMT_VMATREE_HPP
+#include "nmt/memTag.hpp"
#include "nmt/memTag.hpp"
#include "nmt/nmtNativeCallStackStorage.hpp"
#include "nmt/nmtTreap.hpp"
@@ -40,10 +41,12 @@
// The set of points is stored in a balanced binary tree for efficient querying and updating.
class VMATree {
friend class NMTVMATreeTest;
+ friend class VMTWithVMATreeTest;
// A position in memory.
public:
using position = size_t;
using size = size_t;
+ using SIndex = NativeCallStackStorage::StackIndex;
class PositionComparator {
public:
@@ -55,27 +58,29 @@ public:
}
};
- enum class StateType : uint8_t { Reserved, Committed, Released, LAST };
+ // Bit fields view: bit 0 for Reserved, bit 1 for Committed.
+ // Setting a region as Committed preserves the Reserved state.
+ enum class StateType : uint8_t { Reserved = 1, Committed = 3, Released = 0, st_number_of_states = 4 };
private:
- static const char* statetype_strings[static_cast(StateType::LAST)];
+ static const char* statetype_strings[static_cast(StateType::st_number_of_states)];
public:
NONCOPYABLE(VMATree);
static const char* statetype_to_string(StateType type) {
- assert(type != StateType::LAST, "must be");
+ assert(type < StateType::st_number_of_states, "must be");
return statetype_strings[static_cast(type)];
}
// Each point has some stack and a tag associated with it.
struct RegionData {
- const NativeCallStackStorage::StackIndex stack_idx;
+ const SIndex stack_idx;
const MemTag mem_tag;
RegionData() : stack_idx(), mem_tag(mtNone) {}
- RegionData(NativeCallStackStorage::StackIndex stack_idx, MemTag mem_tag)
+ RegionData(SIndex stack_idx, MemTag mem_tag)
: stack_idx(stack_idx), mem_tag(mem_tag) {}
static bool equals(const RegionData& a, const RegionData& b) {
@@ -91,15 +96,27 @@ private:
private:
// Store the type and mem_tag as two bytes
uint8_t type_tag[2];
- NativeCallStackStorage::StackIndex sidx;
+ NativeCallStackStorage::StackIndex _reserved_stack;
+ NativeCallStackStorage::StackIndex _committed_stack;
public:
- IntervalState() : type_tag{0,0}, sidx() {}
+ IntervalState() : type_tag{0,0}, _reserved_stack(NativeCallStackStorage::invalid), _committed_stack(NativeCallStackStorage::invalid) {}
+ IntervalState(const StateType type,
+ const MemTag mt,
+ const NativeCallStackStorage::StackIndex res_stack,
+ const NativeCallStackStorage::StackIndex com_stack) {
+ assert(!(type == StateType::Released) || mt == mtNone, "Released state-type must have memory tag mtNone");
+ type_tag[0] = static_cast(type);
+ type_tag[1] = static_cast(mt);
+ _reserved_stack = res_stack;
+ _committed_stack = com_stack;
+ }
IntervalState(const StateType type, const RegionData data) {
assert(!(type == StateType::Released) || data.mem_tag == mtNone, "Released state-type must have memory tag mtNone");
type_tag[0] = static_cast(type);
type_tag[1] = static_cast(data.mem_tag);
- sidx = data.stack_idx;
+ _reserved_stack = data.stack_idx;
+ _committed_stack = NativeCallStackStorage::invalid;
}
StateType type() const {
@@ -110,16 +127,50 @@ private:
return static_cast(type_tag[1]);
}
- RegionData regiondata() const {
- return RegionData{sidx, mem_tag()};
+ RegionData reserved_regiondata() const {
+ return RegionData{_reserved_stack, mem_tag()};
+ }
+ RegionData committed_regiondata() const {
+ return RegionData{_committed_stack, mem_tag()};
}
void set_tag(MemTag tag) {
type_tag[1] = static_cast(tag);
}
- NativeCallStackStorage::StackIndex stack() const {
- return sidx;
+ NativeCallStackStorage::StackIndex reserved_stack() const {
+ return _reserved_stack;
+ }
+
+ NativeCallStackStorage::StackIndex committed_stack() const {
+ return _committed_stack;
+ }
+
+ void set_reserve_stack(NativeCallStackStorage::StackIndex idx) {
+ _reserved_stack = idx;
+ }
+
+ void set_commit_stack(NativeCallStackStorage::StackIndex idx) {
+ _committed_stack = idx;
+ }
+
+ bool has_reserved_stack() {
+ return _reserved_stack != NativeCallStackStorage::invalid;
+ }
+
+ bool has_committed_stack() {
+ return _committed_stack != NativeCallStackStorage::invalid;
+ }
+
+ void set_type(StateType t) {
+ type_tag[0] = static_cast(t);
+ }
+
+ bool equals(const IntervalState& other) const {
+ return mem_tag() == other.mem_tag() &&
+ type() == other.type() &&
+ reserved_stack() == other.reserved_stack() &&
+ committed_stack() == other.committed_stack();
}
};
@@ -130,8 +181,14 @@ private:
IntervalState out;
bool is_noop() {
+ if (in.type() == StateType::Released &&
+ in.type() == out.type() &&
+ in.mem_tag() == out.mem_tag()) {
+ return true;
+ }
return in.type() == out.type() &&
- RegionData::equals(in.regiondata(), out.regiondata());
+ RegionData::equals(in.reserved_regiondata(), out.reserved_regiondata()) &&
+ RegionData::equals(in.committed_regiondata(), out.committed_regiondata());
}
};
@@ -193,8 +250,44 @@ public:
#endif
};
+ enum Operation {Release, Reserve, Commit, Uncommit};
+ struct RequestInfo {
+ position A, B;
+ StateType _op;
+ MemTag tag;
+ SIndex callstack;
+ bool use_tag_inplace;
+ Operation op() const {
+ return
+ _op == StateType::Reserved && !use_tag_inplace ? Operation::Reserve :
+ _op == StateType::Committed ? Operation::Commit :
+ _op == StateType::Reserved && use_tag_inplace ? Operation::Uncommit :
+ Operation::Release;
+ }
+
+ int op_to_index() const {
+ return
+ _op == StateType::Reserved && !use_tag_inplace ? 1 :
+ _op == StateType::Committed ? 2 :
+ _op == StateType::Reserved && use_tag_inplace ? 3 :
+ 0;
+ }
+ };
+
private:
SummaryDiff register_mapping(position A, position B, StateType state, const RegionData& metadata, bool use_tag_inplace = false);
+ StateType get_new_state(const StateType existinting_state, const RequestInfo& req) const;
+ MemTag get_new_tag(const MemTag existinting_tag, const RequestInfo& req) const;
+ SIndex get_new_reserve_callstack(const SIndex existinting_stack, const StateType ex, const RequestInfo& req) const;
+ SIndex get_new_commit_callstack(const SIndex existinting_stack, const StateType ex, const RequestInfo& req) const;
+ void compute_summary_diff(const SingleDiff::delta region_size, const MemTag t1, const StateType& ex, const RequestInfo& req, const MemTag new_tag, SummaryDiff& diff) const;
+ void update_region(TreapNode* n1, TreapNode* n2, const RequestInfo& req, SummaryDiff& diff);
+ int state_to_index(const StateType st) const {
+ return
+ st == StateType::Released ? 0 :
+ st == StateType::Reserved ? 1 :
+ st == StateType::Committed ? 2 : -1;
+ }
public:
SummaryDiff reserve_mapping(position from, size size, const RegionData& metadata) {
@@ -215,12 +308,8 @@ public:
return register_mapping(from, from + size, StateType::Reserved, metadata, true);
}
- SummaryDiff release_mapping(position from, size size) {
- return register_mapping(from, from + size, StateType::Released, VMATree::empty_regiondata);
- }
-
- VMATreap& tree() {
- return _tree;
+ SummaryDiff release_mapping(position from, position sz) {
+ return register_mapping(from, from + sz, StateType::Released, VMATree::empty_regiondata);
}
public:
@@ -232,7 +321,10 @@ public:
#ifdef ASSERT
void print_on(outputStream* out);
#endif
-
+ template
+ void visit_range_in_order(const position& from, const position& to, F f) {
+ _tree.visit_range_in_order(from, to, f);
+ }
+ VMATreap& tree() { return _tree; }
};
-
#endif
diff --git a/src/hotspot/share/oops/compressedKlass.hpp b/src/hotspot/share/oops/compressedKlass.hpp
index 8666d4409e8..a8d79ab838e 100644
--- a/src/hotspot/share/oops/compressedKlass.hpp
+++ b/src/hotspot/share/oops/compressedKlass.hpp
@@ -214,7 +214,7 @@ public:
// Can only be used after initialization
static address base() { check_init(_base); return _base; }
- static address base_addr() { return (address)&_base; }
+ static address base_addr() { return (address)&_base; }
static int shift() { check_init(_shift); return _shift; }
static address klass_range_start() { return _klass_range_start; }
diff --git a/src/hotspot/share/oops/constantPool.cpp b/src/hotspot/share/oops/constantPool.cpp
index 5b0ee298ddc..3223c56628e 100644
--- a/src/hotspot/share/oops/constantPool.cpp
+++ b/src/hotspot/share/oops/constantPool.cpp
@@ -1937,18 +1937,20 @@ int ConstantPool::find_matching_entry(int pattern_i,
// Compare this constant pool's bootstrap specifier at idx1 to the constant pool
// cp2's bootstrap specifier at idx2.
bool ConstantPool::compare_operand_to(int idx1, const constantPoolHandle& cp2, int idx2) {
- int k1 = operand_bootstrap_method_ref_index_at(idx1);
- int k2 = cp2->operand_bootstrap_method_ref_index_at(idx2);
+ BSMAttributeEntry* e1 = bsm_attribute_entry(idx1);
+ BSMAttributeEntry* e2 = cp2->bsm_attribute_entry(idx2);
+ int k1 = e1->bootstrap_method_index();
+ int k2 = e2->bootstrap_method_index();
bool match = compare_entry_to(k1, cp2, k2);
if (!match) {
return false;
}
- int argc = operand_argument_count_at(idx1);
- if (argc == cp2->operand_argument_count_at(idx2)) {
+ int argc = e1->argument_count();
+ if (argc == e2->argument_count()) {
for (int j = 0; j < argc; j++) {
- k1 = operand_argument_index_at(idx1, j);
- k2 = cp2->operand_argument_index_at(idx2, j);
+ k1 = e1->argument_index(j);
+ k2 = e2->argument_index(j);
match = compare_entry_to(k1, cp2, k2);
if (!match) {
return false;
diff --git a/src/hotspot/share/oops/constantPool.hpp b/src/hotspot/share/oops/constantPool.hpp
index cc9491d7935..be4a7a474d4 100644
--- a/src/hotspot/share/oops/constantPool.hpp
+++ b/src/hotspot/share/oops/constantPool.hpp
@@ -77,6 +77,43 @@ public:
}
};
+class BSMAttributeEntry {
+ friend class ConstantPool;
+ u2 _bootstrap_method_index;
+ u2 _argument_count;
+
+ // The argument indexes are stored right after the object, in a contiguous array.
+ // [ bsmi_0 argc_0 arg_00 arg_01 ... arg_0N bsmi_1 argc_1 arg_10 ... arg_1N ... ]
+ // So in order to find the argument array, jump over ourselves.
+ const u2* argument_indexes() const {
+ return reinterpret_cast(this + 1);
+ }
+ u2* argument_indexes() {
+ return reinterpret_cast(this + 1);
+ }
+ // These are overlays on top of the operands array. Do not construct.
+ BSMAttributeEntry() = delete;
+
+public:
+ // Offsets for SA
+ enum {
+ _bsmi_offset = 0,
+ _argc_offset = 1,
+ _argv_offset = 2
+ };
+
+ int bootstrap_method_index() const {
+ return _bootstrap_method_index;
+ }
+ int argument_count() const {
+ return _argument_count;
+ }
+ int argument_index(int n) const {
+ assert(checked_cast(n) < _argument_count, "oob");
+ return argument_indexes()[n];
+ }
+};
+
class ConstantPool : public Metadata {
friend class VMStructs;
friend class JVMCIVMStructs;
@@ -519,10 +556,6 @@ class ConstantPool : public Metadata {
assert(tag_at(cp_index).has_bootstrap(), "Corrupted constant pool");
return extract_low_short_from_int(*int_at_addr(cp_index));
}
- int bootstrap_operand_base(int cp_index) {
- int bsms_attribute_index = bootstrap_methods_attribute_index(cp_index);
- return operand_offset_at(operands(), bsms_attribute_index);
- }
// The first part of the operands array consists of an index into the second part.
// Extract a 32-bit index value from the first part.
static int operand_offset_at(Array* operands, int bsms_attribute_index) {
@@ -560,47 +593,26 @@ class ConstantPool : public Metadata {
else
return operand_offset_at(operands, nextidx);
}
- int bootstrap_operand_limit(int cp_index) {
- int bsms_attribute_index = bootstrap_methods_attribute_index(cp_index);
- return operand_limit_at(operands(), bsms_attribute_index);
- }
#endif //ASSERT
- // Layout of InvokeDynamic and Dynamic bootstrap method specifier
- // data in second part of operands array. This encodes one record in
- // the BootstrapMethods attribute. The whole specifier also includes
- // the name and type information from the main constant pool entry.
- enum {
- _indy_bsm_offset = 0, // CONSTANT_MethodHandle bsm
- _indy_argc_offset = 1, // u2 argc
- _indy_argv_offset = 2 // u2 argv[argc]
- };
-
// These functions are used in RedefineClasses for CP merge
-
int operand_offset_at(int bsms_attribute_index) {
assert(0 <= bsms_attribute_index &&
bsms_attribute_index < operand_array_length(operands()),
"Corrupted CP operands");
return operand_offset_at(operands(), bsms_attribute_index);
}
- u2 operand_bootstrap_method_ref_index_at(int bsms_attribute_index) {
+
+ BSMAttributeEntry* bsm_attribute_entry(int bsms_attribute_index) {
int offset = operand_offset_at(bsms_attribute_index);
- return operands()->at(offset + _indy_bsm_offset);
- }
- u2 operand_argument_count_at(int bsms_attribute_index) {
- int offset = operand_offset_at(bsms_attribute_index);
- u2 argc = operands()->at(offset + _indy_argc_offset);
- return argc;
- }
- u2 operand_argument_index_at(int bsms_attribute_index, int j) {
- int offset = operand_offset_at(bsms_attribute_index);
- return operands()->at(offset + _indy_argv_offset + j);
+ return reinterpret_cast(operands()->adr_at(offset));
}
+
int operand_next_offset_at(int bsms_attribute_index) {
- int offset = operand_offset_at(bsms_attribute_index) + _indy_argv_offset
- + operand_argument_count_at(bsms_attribute_index);
- return offset;
+ BSMAttributeEntry* bsme = bsm_attribute_entry(bsms_attribute_index);
+ u2* argv_start = bsme->argument_indexes();
+ int offset = argv_start - operands()->data();
+ return offset + bsme->argument_count();
}
// Compare a bootstrap specifier data in the operands arrays
bool compare_operand_to(int bsms_attribute_index1, const constantPoolHandle& cp2,
@@ -617,23 +629,19 @@ class ConstantPool : public Metadata {
u2 bootstrap_method_ref_index_at(int cp_index) {
assert(tag_at(cp_index).has_bootstrap(), "Corrupted constant pool");
- int op_base = bootstrap_operand_base(cp_index);
- return operands()->at(op_base + _indy_bsm_offset);
+ int bsmai = bootstrap_methods_attribute_index(cp_index);
+ return bsm_attribute_entry(bsmai)->bootstrap_method_index();
}
u2 bootstrap_argument_count_at(int cp_index) {
assert(tag_at(cp_index).has_bootstrap(), "Corrupted constant pool");
- int op_base = bootstrap_operand_base(cp_index);
- u2 argc = operands()->at(op_base + _indy_argc_offset);
- DEBUG_ONLY(int end_offset = op_base + _indy_argv_offset + argc;
- int next_offset = bootstrap_operand_limit(cp_index));
- assert(end_offset == next_offset, "matched ending");
- return argc;
+ int bsmai = bootstrap_methods_attribute_index(cp_index);
+ return bsm_attribute_entry(bsmai)->argument_count();
}
u2 bootstrap_argument_index_at(int cp_index, int j) {
- int op_base = bootstrap_operand_base(cp_index);
- DEBUG_ONLY(int argc = operands()->at(op_base + _indy_argc_offset));
- assert((uint)j < (uint)argc, "oob");
- return operands()->at(op_base + _indy_argv_offset + j);
+ int bsmai = bootstrap_methods_attribute_index(cp_index);
+ BSMAttributeEntry* bsme = bsm_attribute_entry(bsmai);
+ assert((uint)j < (uint)bsme->argument_count(), "oob");
+ return bsm_attribute_entry(bsmai)->argument_index(j);
}
// The following methods (name/signature/klass_ref_at, klass_ref_at_noresolve,
diff --git a/src/hotspot/share/oops/fieldInfo.cpp b/src/hotspot/share/oops/fieldInfo.cpp
index 300b45277ad..d0825ba6df8 100644
--- a/src/hotspot/share/oops/fieldInfo.cpp
+++ b/src/hotspot/share/oops/fieldInfo.cpp
@@ -22,8 +22,11 @@
*
*/
+#include "memory/resourceArea.hpp"
+#include "cds/cdsConfig.hpp"
#include "oops/fieldInfo.inline.hpp"
#include "runtime/atomic.hpp"
+#include "utilities/packedTable.hpp"
void FieldInfo::print(outputStream* os, ConstantPool* cp) {
os->print_cr("index=%d name_index=%d name=%s signature_index=%d signature=%s offset=%d "
@@ -37,8 +40,10 @@ void FieldInfo::print(outputStream* os, ConstantPool* cp) {
field_flags().as_uint(),
initializer_index(),
generic_signature_index(),
- _field_flags.is_injected() ? lookup_symbol(generic_signature_index())->as_utf8() : cp->symbol_at(generic_signature_index())->as_utf8(),
- contended_group());
+ _field_flags.is_generic() ? (_field_flags.is_injected() ?
+ lookup_symbol(generic_signature_index())->as_utf8() : cp->symbol_at(generic_signature_index())->as_utf8()
+ ) : "",
+ is_contended() ? contended_group() : 0);
}
void FieldInfo::print_from_growable_array(outputStream* os, GrowableArray* array, ConstantPool* cp) {
@@ -62,13 +67,17 @@ Array* FieldInfoStream::create_FieldInfoStream(GrowableArray* fie
StreamSizer s;
StreamFieldSizer sizer(&s);
+ assert(fields->length() == java_fields + injected_fields, "must be");
+
sizer.consumer()->accept_uint(java_fields);
sizer.consumer()->accept_uint(injected_fields);
for (int i = 0; i < fields->length(); i++) {
FieldInfo* fi = fields->adr_at(i);
sizer.map_field_info(*fi);
}
- int storage_size = sizer.consumer()->position() + 1;
+ // Originally there was an extra byte with 0 terminating the reading;
+ // now we check limits instead.
+ int storage_size = sizer.consumer()->position();
Array* const fis = MetadataFactory::new_array(loader_data, storage_size, CHECK_NULL);
using StreamWriter = UNSIGNED5::Writer*, int, ArrayHelper*, int>>;
@@ -79,15 +88,14 @@ Array* FieldInfoStream::create_FieldInfoStream(GrowableArray* fie
writer.consumer()->accept_uint(java_fields);
writer.consumer()->accept_uint(injected_fields);
for (int i = 0; i < fields->length(); i++) {
- FieldInfo* fi = fields->adr_at(i);
- writer.map_field_info(*fi);
+ writer.map_field_info(fields->at(i));
}
#ifdef ASSERT
FieldInfoReader r(fis);
- int jfc = r.next_uint();
+ int jfc, ifc;
+ r.read_field_counts(&jfc, &ifc);
assert(jfc == java_fields, "Must be");
- int ifc = r.next_uint();
assert(ifc == injected_fields, "Must be");
for (int i = 0; i < jfc + ifc; i++) {
FieldInfo fi;
@@ -113,30 +121,221 @@ Array* FieldInfoStream::create_FieldInfoStream(GrowableArray* fie
return fis;
}
-GrowableArray* FieldInfoStream::create_FieldInfoArray(const Array* fis, int* java_fields_count, int* injected_fields_count) {
- int length = FieldInfoStream::num_total_fields(fis);
- GrowableArray* array = new GrowableArray(length);
+int FieldInfoStream::compare_name_and_sig(const Symbol* n1, const Symbol* s1, const Symbol* n2, const Symbol* s2) {
+ int cmp = n1->fast_compare(n2);
+ return cmp != 0 ? cmp : s1->fast_compare(s2);
+}
+
+
+// We use both name and signature during the comparison; while JLS require unique
+// names for fields, JVMS requires only unique name + signature combination.
+struct field_pos {
+ Symbol* _name;
+ Symbol* _signature;
+ int _index;
+ int _position;
+};
+
+class FieldInfoSupplier: public PackedTableBuilder::Supplier {
+ const field_pos* _positions;
+ size_t _elements;
+
+public:
+ FieldInfoSupplier(const field_pos* positions, size_t elements): _positions(positions), _elements(elements) {}
+
+ bool next(uint32_t* key, uint32_t* value) override {
+ if (_elements == 0) {
+ return false;
+ }
+ *key = _positions->_position;
+ *value = _positions->_index;
+ ++_positions;
+ --_elements;
+ return true;
+ }
+};
+
+Array* FieldInfoStream::create_search_table(ConstantPool* cp, const Array* fis, ClassLoaderData* loader_data, TRAPS) {
+ if (CDSConfig::is_dumping_dynamic_archive()) {
+ // We cannot use search table; in case of dynamic archives it should be sorted by "requested" addresses,
+ // but Symbol* addresses are coming from _constants, which has "buffered" addresses.
+ // For background, see new comments inside allocate_node_impl in symbolTable.cpp
+ return nullptr;
+ }
+
FieldInfoReader r(fis);
- *java_fields_count = r.next_uint();
- *injected_fields_count = r.next_uint();
+ int java_fields;
+ int injected_fields;
+ r.read_field_counts(&java_fields, &injected_fields);
+ assert(java_fields >= 0, "must be");
+ if (java_fields == 0 || fis->length() == 0 || static_cast(java_fields) < BinarySearchThreshold) {
+ return nullptr;
+ }
+
+ ResourceMark rm;
+ field_pos* positions = NEW_RESOURCE_ARRAY(field_pos, java_fields);
+ for (int i = 0; i < java_fields; ++i) {
+ assert(r.has_next(), "number of fields must match");
+
+ positions[i]._position = r.position();
+ FieldInfo fi;
+ r.read_field_info(fi);
+
+ positions[i]._name = fi.name(cp);
+ positions[i]._signature = fi.signature(cp);
+ positions[i]._index = i;
+ }
+ auto compare_pair = [](const void* v1, const void* v2) {
+ const field_pos* p1 = reinterpret_cast(v1);
+ const field_pos* p2 = reinterpret_cast(v2);
+ return compare_name_and_sig(p1->_name, p1->_signature, p2->_name, p2->_signature);
+ };
+ qsort(positions, java_fields, sizeof(field_pos), compare_pair);
+
+ PackedTableBuilder builder(fis->length() - 1, java_fields - 1);
+ Array* table = MetadataFactory::new_array(loader_data, java_fields * builder.element_bytes(), CHECK_NULL);
+ FieldInfoSupplier supplier(positions, java_fields);
+ builder.fill(table->data(), static_cast(table->length()), supplier);
+ return table;
+}
+
+GrowableArray* FieldInfoStream::create_FieldInfoArray(const Array* fis, int* java_fields_count, int* injected_fields_count) {
+ FieldInfoReader r(fis);
+ r.read_field_counts(java_fields_count, injected_fields_count);
+ int length = *java_fields_count + *injected_fields_count;
+
+ GrowableArray* array = new GrowableArray(length);
while (r.has_next()) {
FieldInfo fi;
r.read_field_info(fi);
array->append(fi);
}
assert(array->length() == length, "Must be");
- assert(array->length() == *java_fields_count + *injected_fields_count, "Must be");
return array;
}
void FieldInfoStream::print_from_fieldinfo_stream(Array* fis, outputStream* os, ConstantPool* cp) {
- int length = FieldInfoStream::num_total_fields(fis);
FieldInfoReader r(fis);
- int java_field_count = r.next_uint();
- int injected_fields_count = r.next_uint();
+ int java_fields_count;
+ int injected_fields_count;
+ r.read_field_counts(&java_fields_count, &injected_fields_count);
while (r.has_next()) {
FieldInfo fi;
r.read_field_info(fi);
fi.print(os, cp);
}
}
+
+class FieldInfoComparator: public PackedTableLookup::Comparator {
+ const FieldInfoReader* _reader;
+ ConstantPool* _cp;
+ const Symbol* _name;
+ const Symbol* _signature;
+
+public:
+ FieldInfoComparator(const FieldInfoReader* reader, ConstantPool* cp, const Symbol* name, const Symbol* signature):
+ _reader(reader), _cp(cp), _name(name), _signature(signature) {}
+
+ int compare_to(uint32_t position) override {
+ FieldInfoReader r2(*_reader);
+ r2.set_position_and_next_index(position, -1);
+ u2 name_index, sig_index;
+ r2.read_name_and_signature(&name_index, &sig_index);
+ Symbol* mid_name = _cp->symbol_at(name_index);
+ Symbol* mid_sig = _cp->symbol_at(sig_index);
+
+ return FieldInfoStream::compare_name_and_sig(_name, _signature, mid_name, mid_sig);
+ }
+
+#ifdef ASSERT
+ void reset(uint32_t position) override {
+ FieldInfoReader r2(*_reader);
+ r2.set_position_and_next_index(position, -1);
+ u2 name_index, signature_index;
+ r2.read_name_and_signature(&name_index, &signature_index);
+ _name = _cp->symbol_at(name_index);
+ _signature = _cp->symbol_at(signature_index);
+ }
+#endif // ASSERT
+};
+
+#ifdef ASSERT
+void FieldInfoStream::validate_search_table(ConstantPool* cp, const Array* fis, const Array* search_table) {
+ if (search_table == nullptr) {
+ return;
+ }
+ FieldInfoReader reader(fis);
+ int java_fields, injected_fields;
+ reader.read_field_counts(&java_fields, &injected_fields);
+ assert(java_fields > 0, "must be");
+
+ PackedTableLookup lookup(fis->length() - 1, java_fields - 1, search_table);
+ assert(lookup.element_bytes() * java_fields == static_cast(search_table->length()), "size does not match");
+
+ FieldInfoComparator comparator(&reader, cp, nullptr, nullptr);
+ // Check 1: assert that elements have the correct order based on the comparison function
+ lookup.validate_order(comparator);
+
+ // Check 2: Iterate through the original stream (not just search_table) and try if lookup works as expected
+ reader.set_position_and_next_index(0, 0);
+ reader.read_field_counts(&java_fields, &injected_fields);
+ while (reader.has_next()) {
+ int field_start = reader.position();
+ FieldInfo fi;
+ reader.read_field_info(fi);
+ if (fi.field_flags().is_injected()) {
+ // checking only java fields that precede injected ones
+ break;
+ }
+
+ FieldInfoReader r2(fis);
+ int index = r2.search_table_lookup(search_table, fi.name(cp), fi.signature(cp), cp, java_fields);
+ assert(index == static_cast(fi.index()), "wrong index: %d != %u", index, fi.index());
+ assert(index == r2.next_index(), "index should match");
+ assert(field_start == r2.position(), "must find the same position");
+ }
+}
+#endif // ASSERT
+
+void FieldInfoStream::print_search_table(outputStream* st, ConstantPool* cp, const Array* fis, const Array* search_table) {
+ if (search_table == nullptr) {
+ return;
+ }
+ FieldInfoReader reader(fis);
+ int java_fields, injected_fields;
+ reader.read_field_counts(&java_fields, &injected_fields);
+ assert(java_fields > 0, "must be");
+ PackedTableLookup lookup(fis->length() - 1, java_fields - 1, search_table);
+ auto printer = [&] (size_t offset, uint32_t position, uint32_t index) {
+ reader.set_position_and_next_index(position, -1);
+ u2 name_index, sig_index;
+ reader.read_name_and_signature(&name_index, &sig_index);
+ Symbol* name = cp->symbol_at(name_index);
+ Symbol* sig = cp->symbol_at(sig_index);
+ st->print(" [%zu] #%d,#%d = ", offset, name_index, sig_index);
+ name->print_symbol_on(st);
+ st->print(":");
+ sig->print_symbol_on(st);
+ st->print(" @ %p,%p", name, sig);
+ st->cr();
+ };
+
+ lookup.iterate(printer);
+}
+
+int FieldInfoReader::search_table_lookup(const Array* search_table, const Symbol* name, const Symbol* signature, ConstantPool* cp, int java_fields) {
+ assert(java_fields >= 0, "must be");
+ if (java_fields == 0) {
+ return -1;
+ }
+ FieldInfoComparator comp(this, cp, name, signature);
+ PackedTableLookup lookup(_r.limit() - 1, java_fields - 1, search_table);
+ uint32_t position;
+ static_assert(sizeof(uint32_t) == sizeof(_next_index), "field size assert");
+ if (lookup.search(comp, &position, reinterpret_cast(&_next_index))) {
+ _r.set_position(static_cast(position));
+ return _next_index;
+ } else {
+ return -1;
+ }
+}
diff --git a/src/hotspot/share/oops/fieldInfo.hpp b/src/hotspot/share/oops/fieldInfo.hpp
index 8740d539c8f..a98895da9cf 100644
--- a/src/hotspot/share/oops/fieldInfo.hpp
+++ b/src/hotspot/share/oops/fieldInfo.hpp
@@ -222,29 +222,28 @@ public:
void map_field_info(const FieldInfo& fi);
};
-
// Gadget for decoding and reading the stream of field records.
class FieldInfoReader {
- friend class FieldInfoStream;
- friend class ClassFileParser;
- friend class FieldStreamBase;
- friend class FieldInfo;
-
UNSIGNED5::Reader _r;
int _next_index;
- public:
+public:
FieldInfoReader(const Array* fi);
- private:
- uint32_t next_uint() { return _r.next_uint(); }
+private:
+ inline uint32_t next_uint() { return _r.next_uint(); }
void skip(int n) { int s = _r.try_skip(n); assert(s == n,""); }
public:
- int has_next() { return _r.has_next(); }
- int position() { return _r.position(); }
- int next_index() { return _next_index; }
+ void read_field_counts(int* java_fields, int* injected_fields);
+ int has_next() const { return _r.position() < _r.limit(); }
+ int position() const { return _r.position(); }
+ int next_index() const { return _next_index; }
+ void read_name_and_signature(u2* name_index, u2* signature_index);
void read_field_info(FieldInfo& fi);
+
+ int search_table_lookup(const Array* search_table, const Symbol* name, const Symbol* signature, ConstantPool* cp, int java_fields);
+
// skip a whole field record, both required and optional bits
FieldInfoReader& skip_field_info();
@@ -271,6 +270,11 @@ class FieldInfoStream : AllStatic {
friend class JavaFieldStream;
friend class FieldStreamBase;
friend class ClassFileParser;
+ friend class FieldInfoReader;
+ friend class FieldInfoComparator;
+
+ private:
+ static int compare_name_and_sig(const Symbol* n1, const Symbol* s1, const Symbol* n2, const Symbol* s2);
public:
static int num_java_fields(const Array* fis);
@@ -278,9 +282,14 @@ class FieldInfoStream : AllStatic {
static int num_total_fields(const Array* fis);
static Array* create_FieldInfoStream(GrowableArray* fields, int java_fields, int injected_fields,
- ClassLoaderData* loader_data, TRAPS);
+ ClassLoaderData* loader_data, TRAPS);
+ static Array* create_search_table(ConstantPool* cp, const Array* fis, ClassLoaderData* loader_data, TRAPS);
static GrowableArray* create_FieldInfoArray(const Array* fis, int* java_fields_count, int* injected_fields_count);
static void print_from_fieldinfo_stream(Array* fis, outputStream* os, ConstantPool* cp);
+
+ DEBUG_ONLY(static void validate_search_table(ConstantPool* cp, const Array* fis, const Array* search_table);)
+
+ static void print_search_table(outputStream* st, ConstantPool* cp, const Array* fis, const Array* search_table);
};
class FieldStatus {
diff --git a/src/hotspot/share/oops/fieldInfo.inline.hpp b/src/hotspot/share/oops/fieldInfo.inline.hpp
index d3d4d765081..842393729b2 100644
--- a/src/hotspot/share/oops/fieldInfo.inline.hpp
+++ b/src/hotspot/share/oops/fieldInfo.inline.hpp
@@ -56,16 +56,27 @@ inline Symbol* FieldInfo::lookup_symbol(int symbol_index) const {
inline int FieldInfoStream::num_injected_java_fields(const Array* fis) {
FieldInfoReader fir(fis);
- fir.skip(1);
- return fir.next_uint();
+ int java_fields_count;
+ int injected_fields_count;
+ fir.read_field_counts(&java_fields_count, &injected_fields_count);
+ return injected_fields_count;
}
inline int FieldInfoStream::num_total_fields(const Array* fis) {
FieldInfoReader fir(fis);
- return fir.next_uint() + fir.next_uint();
+ int java_fields_count;
+ int injected_fields_count;
+ fir.read_field_counts(&java_fields_count, &injected_fields_count);
+ return java_fields_count + injected_fields_count;
}
-inline int FieldInfoStream::num_java_fields(const Array* fis) { return FieldInfoReader(fis).next_uint(); }
+inline int FieldInfoStream::num_java_fields(const Array* fis) {
+ FieldInfoReader fir(fis);
+ int java_fields_count;
+ int injected_fields_count;
+ fir.read_field_counts(&java_fields_count, &injected_fields_count);
+ return java_fields_count;
+}
template
inline void Mapper::map_field_info(const FieldInfo& fi) {
@@ -94,13 +105,22 @@ inline void Mapper::map_field_info(const FieldInfo& fi) {
inline FieldInfoReader::FieldInfoReader(const Array* fi)
- : _r(fi->data(), 0),
+ : _r(fi->data(), fi->length()),
_next_index(0) { }
+inline void FieldInfoReader::read_field_counts(int* java_fields, int* injected_fields) {
+ *java_fields = next_uint();
+ *injected_fields = next_uint();
+}
+
+inline void FieldInfoReader::read_name_and_signature(u2* name_index, u2* signature_index) {
+ *name_index = checked_cast(next_uint());
+ *signature_index = checked_cast(next_uint());
+}
+
inline void FieldInfoReader::read_field_info(FieldInfo& fi) {
fi._index = _next_index++;
- fi._name_index = checked_cast(next_uint());
- fi._signature_index = checked_cast(next_uint());
+ read_name_and_signature(&fi._name_index, &fi._signature_index);
fi._offset = next_uint();
fi._access_flags = AccessFlags(checked_cast(next_uint()));
fi._field_flags = FieldInfo::FieldFlags(next_uint());
diff --git a/src/hotspot/share/oops/fieldStreams.hpp b/src/hotspot/share/oops/fieldStreams.hpp
index a1c5d77eeb6..0ae828d73d9 100644
--- a/src/hotspot/share/oops/fieldStreams.hpp
+++ b/src/hotspot/share/oops/fieldStreams.hpp
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2011, 2023, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@@ -56,17 +56,23 @@ class FieldStreamBase : public StackObj {
inline FieldStreamBase(const Array* fieldinfo_stream, ConstantPool* constants, int start, int limit);
- inline FieldStreamBase(Array* fieldinfo_stream, ConstantPool* constants);
+ inline FieldStreamBase(const Array* fieldinfo_stream, ConstantPool* constants);
- private:
+ private:
void initialize() {
- int java_fields_count = _reader.next_uint();
- int injected_fields_count = _reader.next_uint();
- assert( _limit <= java_fields_count + injected_fields_count, "Safety check");
+ int java_fields_count;
+ int injected_fields_count;
+ _reader.read_field_counts(&java_fields_count, &injected_fields_count);
+ if (_limit < _index) {
+ _limit = java_fields_count + injected_fields_count;
+ } else {
+ assert( _limit <= java_fields_count + injected_fields_count, "Safety check");
+ }
if (_limit != 0) {
_reader.read_field_info(_fi_buf);
}
}
+
public:
inline FieldStreamBase(InstanceKlass* klass);
@@ -138,8 +144,11 @@ class FieldStreamBase : public StackObj {
// Iterate over only the Java fields
class JavaFieldStream : public FieldStreamBase {
+ Array* _search_table;
+
public:
- JavaFieldStream(const InstanceKlass* k): FieldStreamBase(k->fieldinfo_stream(), k->constants(), 0, k->java_fields_count()) {}
+ JavaFieldStream(const InstanceKlass* k): FieldStreamBase(k->fieldinfo_stream(), k->constants(), 0, k->java_fields_count()),
+ _search_table(k->fieldinfo_search_table()) {}
u2 name_index() const {
assert(!field()->field_flags().is_injected(), "regular only");
@@ -149,7 +158,6 @@ class JavaFieldStream : public FieldStreamBase {
u2 signature_index() const {
assert(!field()->field_flags().is_injected(), "regular only");
return field()->signature_index();
- return -1;
}
u2 generic_signature_index() const {
@@ -164,6 +172,10 @@ class JavaFieldStream : public FieldStreamBase {
assert(!field()->field_flags().is_injected(), "regular only");
return field()->initializer_index();
}
+
+ // Performs either a linear search or binary search through the stream
+ // looking for a matching name/signature combo
+ bool lookup(const Symbol* name, const Symbol* signature);
};
@@ -176,7 +188,6 @@ class InternalFieldStream : public FieldStreamBase {
class AllFieldStream : public FieldStreamBase {
public:
- AllFieldStream(Array* fieldinfo, ConstantPool* constants): FieldStreamBase(fieldinfo, constants) {}
AllFieldStream(const InstanceKlass* k): FieldStreamBase(k->fieldinfo_stream(), k->constants()) {}
};
diff --git a/src/hotspot/share/oops/fieldStreams.inline.hpp b/src/hotspot/share/oops/fieldStreams.inline.hpp
index 776bcd1671c..51faf88c678 100644
--- a/src/hotspot/share/oops/fieldStreams.inline.hpp
+++ b/src/hotspot/share/oops/fieldStreams.inline.hpp
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2025, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@@ -33,22 +33,18 @@
FieldStreamBase::FieldStreamBase(const Array* fieldinfo_stream, ConstantPool* constants, int start, int limit) :
_fieldinfo_stream(fieldinfo_stream),
_reader(FieldInfoReader(_fieldinfo_stream)),
- _constants(constantPoolHandle(Thread::current(), constants)), _index(start) {
- _index = start;
- if (limit < start) {
- _limit = FieldInfoStream::num_total_fields(_fieldinfo_stream);
- } else {
- _limit = limit;
- }
+ _constants(constantPoolHandle(Thread::current(), constants)),
+ _index(start),
+ _limit(limit) {
initialize();
}
-FieldStreamBase::FieldStreamBase(Array* fieldinfo_stream, ConstantPool* constants) :
+FieldStreamBase::FieldStreamBase(const Array* fieldinfo_stream, ConstantPool* constants) :
_fieldinfo_stream(fieldinfo_stream),
_reader(FieldInfoReader(_fieldinfo_stream)),
_constants(constantPoolHandle(Thread::current(), constants)),
_index(0),
- _limit(FieldInfoStream::num_total_fields(_fieldinfo_stream)) {
+ _limit(-1) {
initialize();
}
@@ -57,9 +53,28 @@ FieldStreamBase::FieldStreamBase(InstanceKlass* klass) :
_reader(FieldInfoReader(_fieldinfo_stream)),
_constants(constantPoolHandle(Thread::current(), klass->constants())),
_index(0),
- _limit(FieldInfoStream::num_total_fields(_fieldinfo_stream)) {
+ _limit(-1) {
assert(klass == field_holder(), "");
initialize();
}
+inline bool JavaFieldStream::lookup(const Symbol* name, const Symbol* signature) {
+ if (_search_table != nullptr) {
+ int index = _reader.search_table_lookup(_search_table, name, signature, _constants(), _limit);
+ if (index >= 0) {
+ assert(index < _limit, "must be");
+ _index = index;
+ _reader.read_field_info(_fi_buf);
+ return true;
+ }
+ } else {
+ for (; !done(); next()) {
+ if (this->name() == name && this->signature() == signature) {
+ return true;
+ }
+ }
+ }
+ return false;
+}
+
#endif // SHARE_OOPS_FIELDSTREAMS_INLINE_HPP
diff --git a/src/hotspot/share/oops/instanceKlass.cpp b/src/hotspot/share/oops/instanceKlass.cpp
index 32d7943e1e8..f4ab8c31409 100644
--- a/src/hotspot/share/oops/instanceKlass.cpp
+++ b/src/hotspot/share/oops/instanceKlass.cpp
@@ -686,6 +686,11 @@ void InstanceKlass::deallocate_contents(ClassLoaderData* loader_data) {
}
set_fieldinfo_stream(nullptr);
+ if (fieldinfo_search_table() != nullptr && !fieldinfo_search_table()->is_shared()) {
+ MetadataFactory::free_array(loader_data, fieldinfo_search_table());
+ }
+ set_fieldinfo_search_table(nullptr);
+
if (fields_status() != nullptr && !fields_status()->is_shared()) {
MetadataFactory::free_array(loader_data, fields_status());
}
@@ -1786,13 +1791,12 @@ FieldInfo InstanceKlass::field(int index) const {
}
bool InstanceKlass::find_local_field(Symbol* name, Symbol* sig, fieldDescriptor* fd) const {
- for (JavaFieldStream fs(this); !fs.done(); fs.next()) {
- Symbol* f_name = fs.name();
- Symbol* f_sig = fs.signature();
- if (f_name == name && f_sig == sig) {
- fd->reinitialize(const_cast(this), fs.to_FieldInfo());
- return true;
- }
+ JavaFieldStream fs(this);
+ if (fs.lookup(name, sig)) {
+ assert(fs.name() == name, "name must match");
+ assert(fs.signature() == sig, "signature must match");
+ fd->reinitialize(const_cast(this), fs.to_FieldInfo());
+ return true;
}
return false;
}
@@ -2391,13 +2395,26 @@ jmethodID InstanceKlass::update_jmethod_id(jmethodID* jmeths, Method* method, in
return new_id;
}
+// Allocate the jmethodID cache.
+static jmethodID* create_jmethod_id_cache(size_t size) {
+ jmethodID* jmeths = NEW_C_HEAP_ARRAY(jmethodID, size + 1, mtClass);
+ memset(jmeths, 0, (size + 1) * sizeof(jmethodID));
+ // cache size is stored in element[0], other elements offset by one
+ jmeths[0] = (jmethodID)size;
+ return jmeths;
+}
+
+// When reading outside a lock, use this.
+jmethodID* InstanceKlass::methods_jmethod_ids_acquire() const {
+ return Atomic::load_acquire(&_methods_jmethod_ids);
+}
+
+void InstanceKlass::release_set_methods_jmethod_ids(jmethodID* jmeths) {
+ Atomic::release_store(&_methods_jmethod_ids, jmeths);
+}
+
// Lookup or create a jmethodID.
-// This code is called by the VMThread and JavaThreads so the
-// locking has to be done very carefully to avoid deadlocks
-// and/or other cache consistency problems.
-//
-jmethodID InstanceKlass::get_jmethod_id(const methodHandle& method_h) {
- Method* method = method_h();
+jmethodID InstanceKlass::get_jmethod_id(Method* method) {
int idnum = method->method_idnum();
jmethodID* jmeths = methods_jmethod_ids_acquire();
@@ -2418,15 +2435,12 @@ jmethodID InstanceKlass::get_jmethod_id(const methodHandle& method_h) {
if (jmeths == nullptr) {
MutexLocker ml(JmethodIdCreation_lock, Mutex::_no_safepoint_check_flag);
- jmeths = methods_jmethod_ids_acquire();
+ jmeths = _methods_jmethod_ids;
// Still null?
if (jmeths == nullptr) {
size_t size = idnum_allocated_count();
assert(size > (size_t)idnum, "should already have space");
- jmeths = NEW_C_HEAP_ARRAY(jmethodID, size + 1, mtClass);
- memset(jmeths, 0, (size + 1) * sizeof(jmethodID));
- // cache size is stored in element[0], other elements offset by one
- jmeths[0] = (jmethodID)size;
+ jmeths = create_jmethod_id_cache(size);
jmethodID new_id = update_jmethod_id(jmeths, method, idnum);
// publish jmeths
@@ -2456,10 +2470,7 @@ void InstanceKlass::update_methods_jmethod_cache() {
if (old_size < size + 1) {
// Allocate a larger one and copy entries to the new one.
// They've already been updated to point to new methods where applicable (i.e., not obsolete).
- jmethodID* new_cache = NEW_C_HEAP_ARRAY(jmethodID, size + 1, mtClass);
- memset(new_cache, 0, (size + 1) * sizeof(jmethodID));
- // The cache size is stored in element[0]; the other elements are offset by one.
- new_cache[0] = (jmethodID)size;
+ jmethodID* new_cache = create_jmethod_id_cache(size);
for (int i = 1; i <= (int)old_size; i++) {
new_cache[i] = cache[i];
@@ -2470,23 +2481,29 @@ void InstanceKlass::update_methods_jmethod_cache() {
}
}
-// Figure out how many jmethodIDs haven't been allocated, and make
-// sure space for them is pre-allocated. This makes getting all
-// method ids much, much faster with classes with more than 8
+// Make a jmethodID for all methods in this class. This makes getting all method
+// ids much, much faster with classes with more than 8
// methods, and has a *substantial* effect on performance with jvmti
// code that loads all jmethodIDs for all classes.
-void InstanceKlass::ensure_space_for_methodids(int start_offset) {
- int new_jmeths = 0;
- int length = methods()->length();
- for (int index = start_offset; index < length; index++) {
- Method* m = methods()->at(index);
- jmethodID id = m->find_jmethod_id_or_null();
- if (id == nullptr) {
- new_jmeths++;
- }
+void InstanceKlass::make_methods_jmethod_ids() {
+ MutexLocker ml(JmethodIdCreation_lock, Mutex::_no_safepoint_check_flag);
+ jmethodID* jmeths = _methods_jmethod_ids;
+ if (jmeths == nullptr) {
+ jmeths = create_jmethod_id_cache(idnum_allocated_count());
+ release_set_methods_jmethod_ids(jmeths);
}
- if (new_jmeths != 0) {
- Method::ensure_jmethod_ids(class_loader_data(), new_jmeths);
+
+ int length = methods()->length();
+ for (int index = 0; index < length; index++) {
+ Method* m = methods()->at(index);
+ int idnum = m->method_idnum();
+ assert(!m->is_old(), "should not have old methods or I'm confused");
+ jmethodID id = Atomic::load_acquire(&jmeths[idnum + 1]);
+ if (!m->is_overpass() && // skip overpasses
+ id == nullptr) {
+ id = Method::make_jmethod_id(class_loader_data(), m);
+ Atomic::release_store(&jmeths[idnum + 1], id);
+ }
}
}
@@ -2610,6 +2627,7 @@ void InstanceKlass::metaspace_pointers_do(MetaspaceClosure* it) {
}
it->push(&_fieldinfo_stream);
+ it->push(&_fieldinfo_search_table);
// _fields_status might be written into by Rewriter::scan_method() -> fd.set_has_initialized_final_update()
it->push(&_fields_status, MetaspaceClosure::_writable);
@@ -2710,6 +2728,8 @@ void InstanceKlass::remove_unshareable_info() {
DEBUG_ONLY(_shared_class_load_count = 0);
remove_unshareable_flags();
+
+ DEBUG_ONLY(FieldInfoStream::validate_search_table(_constants, _fieldinfo_stream, _fieldinfo_search_table));
}
void InstanceKlass::remove_unshareable_flags() {
@@ -2816,6 +2836,8 @@ void InstanceKlass::restore_unshareable_info(ClassLoaderData* loader_data, Handl
if (DiagnoseSyncOnValueBasedClasses && has_value_based_class_annotation() && !is_value_based()) {
set_is_value_based();
}
+
+ DEBUG_ONLY(FieldInfoStream::validate_search_table(_constants, _fieldinfo_stream, _fieldinfo_search_table));
}
// Check if a class or any of its supertypes has a version older than 50.
@@ -2914,7 +2936,7 @@ void InstanceKlass::release_C_heap_structures(bool release_sub_metadata) {
JNIid::deallocate(jni_ids());
set_jni_ids(nullptr);
- jmethodID* jmeths = methods_jmethod_ids_acquire();
+ jmethodID* jmeths = _methods_jmethod_ids;
if (jmeths != nullptr) {
release_set_methods_jmethod_ids(nullptr);
FreeHeap(jmeths);
@@ -3492,7 +3514,7 @@ void InstanceKlass::add_osr_nmethod(nmethod* n) {
for (int l = CompLevel_limited_profile; l < n->comp_level(); l++) {
nmethod *inv = lookup_osr_nmethod(n->method(), n->osr_entry_bci(), l, true);
if (inv != nullptr && inv->is_in_use()) {
- inv->make_not_entrant(nmethod::ChangeReason::OSR_invalidation_of_lower_level);
+ inv->make_not_entrant(nmethod::InvalidationReason::OSR_INVALIDATION_OF_LOWER_LEVEL);
}
}
}
@@ -3760,6 +3782,11 @@ void InstanceKlass::print_on(outputStream* st) const {
map++;
}
st->cr();
+
+ if (fieldinfo_search_table() != nullptr) {
+ st->print_cr(BULLET"---- field info search table:");
+ FieldInfoStream::print_search_table(st, _constants, _fieldinfo_stream, _fieldinfo_search_table);
+ }
}
void InstanceKlass::print_value_on(outputStream* st) const {
@@ -4261,17 +4288,14 @@ bool InstanceKlass::should_clean_previous_versions_and_reset() {
return ret;
}
-// This nulls out jmethodIDs for all methods in 'klass'
-// It needs to be called explicitly for all previous versions of a class because these may not be cleaned up
-// during class unloading.
-// We can not use the jmethodID cache associated with klass directly because the 'previous' versions
-// do not have the jmethodID cache filled in. Instead, we need to lookup jmethodID for each method and this
-// is expensive - O(n) for one jmethodID lookup. For all contained methods it is O(n^2).
-// The reason for expensive jmethodID lookup for each method is that there is no direct link between method and jmethodID.
-void InstanceKlass::clear_jmethod_ids(InstanceKlass* klass) {
+// This nulls out the jmethodID for all obsolete methods in the previous version of the 'klass'.
+// These obsolete methods only exist in the previous version and we're about to delete the memory for them.
+// The jmethodID for these are deallocated when we unload the class, so this doesn't remove them from the table.
+void InstanceKlass::clear_obsolete_jmethod_ids(InstanceKlass* klass) {
Array* method_refs = klass->methods();
for (int k = 0; k < method_refs->length(); k++) {
Method* method = method_refs->at(k);
+ // Only need to clear obsolete methods.
if (method != nullptr && method->is_obsolete()) {
method->clear_jmethod_id();
}
@@ -4321,7 +4345,7 @@ void InstanceKlass::purge_previous_version_list() {
// Unlink from previous version list.
assert(pv_node->class_loader_data() == loader_data, "wrong loader_data");
InstanceKlass* next = pv_node->previous_versions();
- clear_jmethod_ids(pv_node); // jmethodID maintenance for the unloaded class
+ clear_obsolete_jmethod_ids(pv_node); // jmethodID maintenance for the unloaded class
pv_node->link_previous_versions(nullptr); // point next to null
last->link_previous_versions(next);
// Delete this node directly. Nothing is referring to it and we don't
diff --git a/src/hotspot/share/oops/instanceKlass.hpp b/src/hotspot/share/oops/instanceKlass.hpp
index 078c5b81841..2512d8d869d 100644
--- a/src/hotspot/share/oops/instanceKlass.hpp
+++ b/src/hotspot/share/oops/instanceKlass.hpp
@@ -276,6 +276,7 @@ class InstanceKlass: public Klass {
// Fields information is stored in an UNSIGNED5 encoded stream (see fieldInfo.hpp)
Array* _fieldinfo_stream;
+ Array* _fieldinfo_search_table;
Array