Review of 'Explainable drug repurposing via path-based knowledge graph completion'

Jimenez et al. present a new method for path-based knowledge graph completion applied to drug repurposing. XG4Repo suggests new paths on the Hetionet data set and is one of the few knowledge graph based tools allowing queries to link drugs to diseases that can be treated with them. The method is evaluated by splitting drug - disease associations into train, test and validation set. The performance is then evaluated based on the mean reciprocal rank of the candidates in the test set and compared to other state of the art tools. The use of XG4Repo is further exemplified for three drugs, where medical literature evidence is presented for supporting the predictions. The manuscript is generally well written and offers mostly sufficient details. More specific comments are found below.

General:

• Some paragraphs are a bit difficult to parse, consider revising the manuscript for clarity and correct grammar.
• Introduction rather short
• The research gap is not very well described. The section "Our contribution" leaves it open e.g. why a general architecture is currently missing or where the authors see a gap in interpretability w.r.t. current methodology.
• Figures are not shown in the order they are first mentioned in the paper, e.g. Figure 11.

Introduction:

• Examples for graph based drug repurposing are given but not described, e.g. ref. 7-9 are not mentioned.
• "These methods cannot capture multistep relations" - please explain in more detail what you mean by that.
• "Most of them are based on paths that relate drugs and diseases through the nodes of the graph". It remains very vague what those paths look like, i.e. how are drugs and diseases linked. Via shared genes / proteins or do you refer to other types of nodes? How does such a path represent a biological explanation? Please elaborate further here.
• Similarly, it is not clear how metapaths differ from the aforementioned paths in this context without further details.
• "Metapaths that are known to be useful" - known how?
• "Other methods do the opposite, evaluating every possible metapath" - please elaborate in more detail what this evaluation could look like.
• "disease modules are used to predict a list of drug candidates to treat a certain disease". It is unclear how this is achieved and what additional data is needed for this.
• "They generated disease modules using network-based medical algorithms". This should be explained earlier in the paragraph (before switching to drug candidate prediction). Also, I think you want to refer to network medicine algorithms rather than network-based medical algorithms.
• "apply some technique to provide explanations." this is again very ambiguous, please elaborate what those techniques are.

Data:

• Data was split into training, test and validation but the splitting was not further described. What measures were taken to avoid data leakage. Did the author consider cold splits where a drug / disease is contained either entirely in the training or in the test set?

Evaluation:

• The evaluation metric is not properly defined here.

Results:

• An articulated strength of the xg4repo approach is interpretability. While the manuscript compares predictions with the medical literature and related databases such as DrugBank, a functional link of drug and disease that explains the prediction is not discussed here. As the authors argue that this is a strength of their approach this should be improved.  For example, epirubicin is a topisomerase-2 inhibitor which leads to double streand breaks in the DNA. The mechanistic link of the drug and the suggested diseases is not obvious at all from the proposed paths and rules.
• Limitations of this approach are not sufficiently discussed. For example, the evaluation here is based on mean reciprocal rank which allows relative comparisons between methods (good methods will rank ground truth diseases higher). However, this is no indication that the results are necessarily trustworthy as suggested in the paper. This is because the number of false positive predictions can not be assessed. In my opinion, this limitation needs to be emphasized more clearly. Another limitation is that the knowledge graph can only indirectly leverage molecular associations which have been previously observed, while methods employing molecular data could validate rules. The approach presented here is also not suited for personalized or precision medicine, as the patient characteristics are not taken into account.

Code:

• The code for XG4Repo is available on github but it lacks any sort of usage documentation and does not even provide a basic readme or usage examples. Furthermore, no open source license has been added making it hard to use this code for further research.