BuddyPress.org

@thesun2012 Use bp_loggedin_user_id() to tie the action of starring or unstarring a message to the user logged in and performing the action. This function returns the ID of the user who is currently logged into the site. When a user is viewing their profile, both bp_displayed_user_id() and bp_loggedin_user_id() will return the same user ID, as the “displayed user” and the “logged-in user” are the same person in this scenario.

@thesun2012 bp_legacy_theme_ajax_messages_star_handler is not mandatory; you can also create your custom handler based on it
for example

function wbcom_custom_buddypress_ajax_handler() {
    check_ajax_referer('custom-bp-ajax-nonce', 'security');

    $displayed_user_id = isset($_POST['displayed_user_id']) && !empty($_POST['displayed_user_id']) 
                         ? (int) sanitize_text_field($_POST['displayed_user_id']) 
                         : get_current_user_id();

    $message_id = isset($_POST['message_id']) ? (int) sanitize_text_field($_POST['message_id']) : 0;
    $star_status = isset($_POST['star_status']) ? sanitize_text_field($_POST['star_status']) : '';
    $bulk = isset($_POST['bulk']) && !empty($_POST['bulk']);

    if (!bp_is_active('messages', 'star') || empty($message_id) || !is_user_logged_in() || !bp_core_can_edit_settings()) {
        wp_send_json_error(array('message' => 'Invalid request or insufficient permissions.'));
        return;
    }

    $result = bp_messages_star_set_action(array(
        'action' => $star_status,
        'message_id' => $message_id,
        'bulk' => $bulk,
        'user_id' => $displayed_user_id
    ));

    if ($result) {
        wp_send_json_success();
    } else {
        wp_send_json_error(array('message' => 'Failed to update message status.'));
    }
}

add_action('wp_ajax_wbcom_custom_buddypress_action', 'wbcom_custom_buddypress_ajax_handler');

AJAX Request Adjustments like this

jQuery.ajax({
    url: ajaxurl,
    type: 'POST',
    data: {
        'action': 'wbcom_custom_buddypress_action',
        'displayed_user_id': displayedUserId, // Include if available
        'message_id': messageId,
        'star_status': starStatus,
        // Additional data...
    },
    success: function(response) {
        // Handle success
    },
    error: function(error) {
        // Handle error
    }
});

We can suggest modifying the bp_legacy_theme_ajax_messages_star_handler() function to handle the displayed user ID and use the logged-in user ID as a fallback.

function bp_legacy_theme_ajax_messages_star_handler() {
    if ( false === bp_is_active( 'messages', 'star' ) || empty( $_POST['message_id'] ) ) {
        return;
    }

    // Check nonce.
    check_ajax_referer( 'bp-messages-star-' . (int) $_POST['message_id'], 'nonce' );

    // Check capability.
    if ( ! is_user_logged_in() || ! bp_core_can_edit_settings() ) {
        return;
    }

    // Get the displayed user ID or fallback to the logged-in user ID.
    $user_id = isset($_POST['displayed_user_id']) && !empty($_POST['displayed_user_id']) 
               ? (int) $_POST['displayed_user_id'] 
               : get_current_user_id();

    if ( true === bp_messages_star_set_action( array(
        'action'     => $_POST['star_status'],
        'message_id' => (int) $_POST['message_id'],
        'bulk'       => ! empty( $_POST['bulk'] ) ? true : false,
        'user_id'    => $user_id  // Use the determined user ID
     ) ) ) {
        echo '1';
        die();
    }

    echo '-1';
    die();
}