Register now After registration you will be able to apply for this opportunity online.
Automated kinematic estimation of the knee joint using deep learning
Knee kinematics is critical for diagnosing pathologies such as osteoarthritis and providing guidance for implant design. Estimating knee kinematics requires aligning a model with a target X-ray image. This estimation process, often implemented by human labor, can be very time-consuming. This research aims to use a deep learning network to estimate the pose (kinematics) from X-ray images, partially replacing manual labor. Such a network should predict a pose from a current fluoroscopic image. By the end of this project, a robust pipeline should be completed, achieving baseline performance to provide convincing pose estimation for images from different modalities (single-plane system & dual-plane system; natural bone model & implant model).
Keywords: X-ray; Computational method; Medical image; Image registration; Rendering.
2D3D pose estimation (also called 2d3d image registration) is an essential step in fluoroscope analysis. Finding an alignment between 3D model rendering and the target X-ray becomes a prerequisite step before obtaining the true pose data for kinematics analysis, such as relative tibiofemoral movement. Previous research in pose estimation involves direct CNN prediction, Point-of-Interest networks, and some deep learning approaches specialized for pose estimation. We are going to start from previous work we did before (CNN+segmentation) and explore more modern approaches in pose estimation. Although most example images in the project description are with a knee implant model, the natural bone model is definitely under our major consideration.
In general, the main goal of this research project falls into two parts: 1. We will build a multi-step deep learning pipeline with the function of contextualization in the first step (segmentation or preprocessing) and direct regression (e.g. to 6D or Quaternion representation) as the second step. The aim is to test the efficacy of different contextualization and regression network structures and choose the best choice for knee joint estimation. 2. We will explore more modern techniques with a network in knee joint estimation. Available choices include a combination with iterative model optimization methods, Point-of-Interest networks, and NeuRF networks in inferring 3D information. The ultimate baseline for deep learning estimation should be within around 10mm for translation prediction error and 10 degrees for rotation prediction error.
2D3D pose estimation (also called 2d3d image registration) is an essential step in fluoroscope analysis. Finding an alignment between 3D model rendering and the target X-ray becomes a prerequisite step before obtaining the true pose data for kinematics analysis, such as relative tibiofemoral movement. Previous research in pose estimation involves direct CNN prediction, Point-of-Interest networks, and some deep learning approaches specialized for pose estimation. We are going to start from previous work we did before (CNN+segmentation) and explore more modern approaches in pose estimation. Although most example images in the project description are with a knee implant model, the natural bone model is definitely under our major consideration.
In general, the main goal of this research project falls into two parts: 1. We will build a multi-step deep learning pipeline with the function of contextualization in the first step (segmentation or preprocessing) and direct regression (e.g. to 6D or Quaternion representation) as the second step. The aim is to test the efficacy of different contextualization and regression network structures and choose the best choice for knee joint estimation. 2. We will explore more modern techniques with a network in knee joint estimation. Available choices include a combination with iterative model optimization methods, Point-of-Interest networks, and NeuRF networks in inferring 3D information. The ultimate baseline for deep learning estimation should be within around 10mm for translation prediction error and 10 degrees for rotation prediction error.
1. Build a deep learning pipeline for knee pose estimation from X-ray images.
2. Test and select the best network structures for knee joint estimation.
3. Achieve a translation prediction error within 10mm and a rotation prediction error within 10 degrees.
1. Build a deep learning pipeline for knee pose estimation from X-ray images. 2. Test and select the best network structures for knee joint estimation. 3. Achieve a translation prediction error within 10mm and a rotation prediction error within 10 degrees.