Computational Forensics

Introduction

Our research theme is Computational Forensics(CF). CF requires joint efforts by forensics and computational scientists with benefits to both. To complement the lack of knowledge among forensic and computer experts, we collaborate with forensics domain experts(from the University of Yamanashi, etc.). The goal of “Computational XXXX” is to realize the intellectual activities that human beings originally wanted to do using computer graphics(CG) and computational resources.

The essential thing in forensic science is to justify the severity of the crime. CF can numerically quantify how much malice the attacker harbored when he attacked the victim, the concept of “malice index.” Malice index would be a quantitative indicator, tailored specially for the jury, calculated from the severity of injuries or other factors such as the angle of the blade. Besides, it also could be used to compare with past precedents to determine the appropriate punishment. In our opinion, this is one of the CF ways forward. 

We are developing efficient user interfaces for investigations, forensic analysis, and sentencing in court, with the concept of x-LMML.

FORSETI: Forensic autopsy system for e-court instruments

Forensic science holds a unique position among all other disciplines due to its important social implications—lying at the intersection of medical science and legal science. Indeed, forensic autopsy reports, which systematically describe autopsy findings, are imperative both for legal medicine and courts. Typically, they are created by medical examiners (MEs) collaboratively working with diagnostic radiologists (DRs). The forensic autopsy reports also serve as underlying legal documents in judicial trials for engaged judicial personnel (JP). A challenging issue is to provide an efficient common computational environment for the MEs and DRs as well as JP. We elaborate on the issue from the viewpoints of legal medicine and courts.

Legal medicine. The fundamental works of MEs and DRs are physical autopsy (PA) and virtual autopsy (VA), respectively. Authoring and referencing are involved in their works. For authoring, dictation and text generation accompanied with diagnostic imaging are commonly used both in PA and in VA. These methods are error-prone for documentation, interaction, and illustration of the autopsy results. As for current referencing, MEs use anatomical worksheets (considered time-consuming) to refer to DRs’ work, while DRs use 3D surface imaging of corpses (requiring expensive equipment) to refer to MEs’ work. Therefore, an efficient way needs to be devised for MEs and DRs to acquire and compare autopsy findings.

Courts. The primary goal of the MEs and DRs is to document comprehensible and reliable autopsy results efficiently, which are then reviewed by various kinds of JP (including investigators, prosecutors, judges, and juries) to generate their own court documents. However, existing autopsy reports are cumbersome for JP (and even for MEs and DRs) to reuse. One reason for this is the lack of spatial inference among autopsy findings, and another is the use of difficult technical terms in forensics.

In our prior research, a browser with authoring tools using a language called legal medicine markup language (LMML) is presented. The proposed prototype assists MEs in effectively generating PA reports, but the research is not targeted at VA reports nor the court documents. In this work, we attempt to address the aforementioned issue by extending the prior research and presenting an integrated visual analysis environment named FORSETI—forensic autopsy system for e-court instrments. The FORSETI system is designed to have two technical essences. The first is to be designed on the basis of an extended version of LMML (x-LMML) to encompass from PA to VA in authoring integrated electronic autopsy (e-autopsy) reports. The second is autopsy juxtaposition, which improves the reliability and efficiency of the autopsy by helping MEs and DRs refer to each other’s findings. A fictitious case with the Visible Female Dataset is used to demonstrate the effectiveness of an initial prototype of the FORSETI system.

 

THEMIS: THeoritical Estimation of Meaning of InSults

THEMIS (Theotretical Estimation of Meaning of InSults) as a sub-system of computational forensics, It is used to stakeholders for analyses and presents similarity between photographs recorded in LMML files.

Mathematical Model of Meaning

The mathematical model of meaning is a method proposed by Professor Kiyoki et al., who is the local of this project, to calculate the semantic similarity and described in 1995 SIGMOD Record. For example, the word “green” has multiple meanings associated with it, but considering that it is a “road”-related context, “green” means “go” It is possible to show that it is close to Furthermore, not only for documents but also for multimedia, it is possible to calculate the semantic similarity regardless of modal by extracting the semantic elements possessed by the content and applying meaning.

Detecting means of wound

In order to apply a mathematical model of meaning to a wound image, THEMIS makes a meaning of the wound. There are two major characteristics of “morphology” and “color” as the items of meaning to be extracted. The outline of the wound is extracted from the wound image by image processing, and the morphological features (whether it is an open wound or a sharp wound) are extracted from the outline. On the other hand, for the color characteristics, the representative color of the wound is extracted using the color system of L*a*b* as the characteristic of the wound. By expressing the extracted features as a vector, similarity calculation is performed using a mathematical model of meaning.

 

Members

 

Name Affiliation Web site
Baoqing Wang Keio university
Yume Asayama Keio university
Masanori Nakayama Keio univercity
Yasushi Kiyoki Keio university
Xiaoyang Mao University of Yamanashi Mao laboratory
Yuriko Takeshima Tokyo university of technology Tokyo University of Technology staff profile
Noboru Adachi University of Yamanashi University of Yamanashi Department of Legal Medicine staff profile
Hideki Shojo University of Yamanashi University of Yamanashi Department of Legal Medicine staff profile

 

Publications

Journals

  1. Baoqing Wang, Noboru Adachi, Issei Fujishiro: “FORSETI: A visual analysis environment enabling provenance awareness for the accountability of e-autopsy reports,” Journal of Visual Informatics, Vol. 6, No. 3, pp. 69-80, September 2022. (pdf)
  2. Baoqing Wang, Yume Asayama, Malik Olivier Boussejra, Hideki Shojo, Noboru Adachi, Issei Fujishiro: “FORSETI: a visual analysis environment for authoring autopsy reports in extended legal medicine mark-up language,” The Visual Computer, Vol. 37, pp. 2951–2963, June 2021. (pdf)

Presentations

  1. BaoqingWang, Noboru Adachi, and Issei Fujishiro, “FORSETI: A provenance-aware visual analysis environment for the lifecycle management of e-autopsy reports,” in Proceedings of the 5th international workshop on Big Data Visual Exploration and Analytics, March 29, 2022. [Link][pdf].
  2. Yume Asayama, Baoqing Wang, Masanori Nakayama, Hideki Shojo, Noburu Adachi, Yasushi Kiyoki, Issei Fujishiro: “THEMIS: Context-sensitive similarity analysis for wound imagery using mathematical model of meaning,” in Proceedings of the 2021 International Conference on Cyberworlds, pp. 128―132, France, September 28―30, 2021. [Link]
  3. Issei Fujishiro, Baoqing Wang, Yume Asayama, Malik Olivier Boussejra, Hideki Shojo, Noboru Adachi, Issei Fujishiro: “FORSETI: a visual analysis environment for authoring autopsy reports in extended legal medicine mark-up language,” invited by the Workshop in Memory of Professor Kunii  in conjunction with CGI 2021, online, September 6, 2021.
  4. Baoqing Wang, Yume Asayama, Issei Fujishiro: “FORSETI and THEMIS: visual analysis environment for computational forensics,” invited by The 4th China-Japan Joint Visualization Workshop, Wuhan, China, online, July 24th, 2021.
  5. Yume Asayama, Baoqing Wang, Masanori Nakayama, Hideki Shojo, Noboru Adachi, Yasushi Kiyoki, Issei Fujishiro: “Visual analysis of wound similarity using mathematical model of meaning,” in Proceedings of the 48th Symposium on Visualization in Japan, Online,  September 24―26, 2020 (in Japanese).
  6. Yume Asayama, Baoqing Wang, Malik Olivier Boussejra, Masanori Nakayama, Hideki Shojo, Noboru Adachi, Yasushi Kiyoki, Issei Fujishiro: “Visual analysis of wound similarity using mathematical model of meaning,” in Proceedings of the 82th National Convention of International Processing Society of Japan, Vol. 2, pp. 155―156, Kanazawa Institute of Technology, Ishikawa, Online, March 5―7, 2016 (in Japanese). Student encouragement award
  7. Malik Olivier Boussejra, Noboru Adachi, Hideki Shojo, Ryohei Takahashi, Issei Fujishiro: “LMML: Developing the environment of the LMML mark-up language for forensic data visualization,” in The Journal of the Institute of Image Electronics Engineers of Japan, vol.45(1), pp. 127, 2016. Malik Olivier Boussejra, Noboru Adachi, Hideki Shojo, Ryohei Takahashi, Issei Fujishiro: “LMML: Developing the environment of the LMML mark-up language for forensic data Visualization,” in The Journal of the Institute of Image Electronics Engineers of Japan, vol.45(1), pp. 127, 2016.
  8. Malik Olivier Boussejra, Noboru Adachi, Hideki Shojo, Ryohei Takahashi, Issei Fujishiro: “LMML: Initial Developments of an Integrated Environment for Forensic Data Visualization,” in EuroVis 2016 – Short Papers, pp. 31－pp. 35, 2016. (pdf)
  9. Malik Olivier Boussejra, Noboru Adachi, Hideki Shojo, Ryohei Takahashi, Issei Fujishiro: “LMML: Describing injuries for forensic data visualization,” in 2016 Nicograph International, pp. 153, 2016. Best poster award (pdf)

 

Grants

 

  1. Grant-in-Aid for Scientific Research (A): 21H04916 (2021—2026)
  2. Grant-in-Aid for Scientific Research (A): 17H00737 (2017—2021)
  3. Grant-in-Aid for Scientific Research (A): 26240015 (2014—2017)

 

Back to AV team page