????AI?ŏd?v?Ȍ???̃x?N?g?????Ƃ́\?\Spring AI?Ńx?N?g???f?[?^?x?[?X?ɑ΂???ގ??x???????????????FSpring AI?Ŏn?߂鐶??AI?v???O???~???O?i4?j

?@?????AEmbeddings Model?𗘗p???āA?^????ꂽ???b?Z?[?W????x?N?g???f?[?^???쐬???Ă݂܂??傤?i???X?g1?j?B

import org.springframework.ai.embedding.EmbeddingModel;
// (?ȗ?)
@ShellMethod(key = "vector-create")
public String create(String message){
  //  (1) EmbeddingModel?̎擾
  EmbeddingModel embeddingModel = context.getBean(EmbeddingModel.class);
  //  (2) ?x?N?g???f?[?^???쐬
  float[] vector = embeddingModel.embed(message);
  return Arrays.toString(vector);
}

?@?i1?j?ŁA?x?N?g???f?[?^???쐬???邽?߂̃??f?????擾???܂??B?C???^?t?F?[?X?N???X?ł???EmbeddingModel???擾???Ă??܂????A???ۂ?OpenAI?ŗ??p????̂?org.springframework.ai.openai.OpenAiEmbeddingModel?N???X?ƂȂ?܂??B

?@???Ƀx?N?g???f?[?^???擾???܂??i2?j?B?x?N?g???f?[?^?̎????̓??f???Ɉˑ????܂????AOpenAI?̏ꍇ?ɂ?1536??????float?Ƃ??ăx?N?g????????܂??B?ȉ??́A???̋?̓I?Ȏ??s??ł??B

shell:>vector-create ???
[0.0149485655, -0.009776097, 0.008102652,.....,-0.0020471597]
?E?E(?ȗ?)

?x?N?g???f?[?^?x?[?X?iVectorStore?j?Ƃ́H

?@?쐬?????x?N?g???f?[?^??ۑ????A?????̃f?[?^?Ƃ̗ގ??x???v?Z?A?Ǘ????邽?߂ɁA?x?N?g???f?[?^?x?[?X???g?p???܂??B???ۂ̃f?[?^?Ǘ??V?X?e???Ƃ??ẮA?N???E?h?T?[?r?X?𗘗p???邱?Ƃ??ł??܂????A?ŋ߂̎?v?ȃ????[?V???i???f?[?^?x?[?X?iRDB?j???x?N?g???f?[?^???T?|?[?g???Ă??܂??B

?@Spring AI?ł́A?}2?̂悤?ȗ???Ńx?N?g???f?[?^?̓o?^?ƌ??????s???܂??B

?}2?@?x?N?g???f?[?^?x?[?X?𗘗p?????lj??ƌ????̊T?O?}

?@?x?N?g???f?[?^?x?[?X???????ꍇ?A??قǏЉ??float[]?^?̃x?N?g???f?[?^?́A???ڃR?[?h??ɂ͌??ꂸ?A?x?N?g???f?[?^?x?[?X?̎??????ɉB??????Ă??܂??B?܂??A???^???Ȃǂ??ꏏ?ɊǗ??ł???悤?ɂȂ?A??????Document?N???X?ŊǗ?????܂??B???̃N???X?́A?x?N?g???????̌??ʂ?List?^?Ŏ擾?ł??܂??B

?@?ǂ̃x?N?g???f?[?^?x?[?X??I?????Ă?VectorStore?C???^?t?F?[?X????ăA?N?Z?X???܂??B?f?[?^?x?[?X?̎?ނɉ????Ď????N???X???p?ӂ???Ă??邽?߁AVectorStore?̃C???X?^???X???쐬????ۂɂ́A???p????f?[?^?x?[?X?ɍ??킹???????N???X???•ʂɎw?肵?A???ꂼ??ɕK?v?Ȑݒ???s???K?v??????܂??i?}3?j?B

?}3?@?x?N?g?????̂??߂̃N???X?T?O

?@?????ɂ?SearchRequest?N???X?𗘗p???܂??B???̃N???X?́A?ǂ̃x?N?g???f?[?^?x?[?X??????I??ł????ʂ??ė??p?ł??܂??B

SimpleVectorStore???p?̂??߂̏???

?@?܂??͊ȒP??SimpleVectorStore?N???X?Ŏ????Ă݂܂??BSimpleVectorStore?̓C?????????Ńf?[?^??ێ????邽?߂̃N???X?ŁAJSON?t?@?C???ł̕ۑ??⃍?[?h???T?|?[?g???Ă??܂??B???p?ɍۂ??đO?????????炸?A?x?N?g???f?[?^?x?[?X?̋@?\???ȈՓI???Ž?y?Ɏ??????Ƃ??ł??܂??B?ŏ??Ƀx?N?g???f?[?^?x?[?X???????ɂ́A?????Â??̃N???X?Ƃ?????ł??傤?B

1. SimpleVectorStore???g?????߂̐ݒ??lj?????

?@SimpleVectorStore???g???ɂ́A?܂??ABuild?t?@?C???ibuild.gradle.kts?j?Ƀ??X?g2?̎w???lj????܂??B

dependencies {
  // (?ȗ?)
  //  ?ȉ???lj?
  implementation("org.springframework.ai:spring-ai-vector-store")
}

?@???̃p?b?P?[?W?lj??ɂ?VectorStore?̊?{?N???X???܂܂?Ă??܂??B??ɏЉ??MariaDB?Ȃǂ𗘗p????ꍇ?ɂ́A???̃p?b?P?[?W?͎????Œlj??????̂Ŏw??͕K?v????܂??񂪁ASimpleVectorStore?݂̂𗘗p????ꍇ?ɂ͎w?肪?K?v?ł??B

2. ?x?N?g???f?[?^?x?[?X?Ƀf?[?^??ۑ??^???[?h????

?@???X?g3?́ASimpleVectorStore?o?R?ŁA?t?@?C???Ƀx?N?g???f?[?^??ۑ??^???[?h???邽?߂̃T???v???ł??B

@ShellComponent
public class VectorCommand {
  private final SimpleVectorStore vectorStore;
  public VectorCommand(ApplicationContext context){
    this.context = context;
    // EmbeddingModel???f???Ǝ擾????
    EmbeddingModel embeddingModel = context.getBean(EmbeddingModel.class);
    //  (1) SimpleVectorStore?̃C???X?^???X???쐬????
    vectorStore = SimpleVectorStore.builder(embeddingModel).build();
  }
  // (?ȗ?)
  // (2) ?t?@?C???Ƀf?[?^??ۑ?????
  @ShellMethod(key = "vector-save")
  public void save(String filename){
    vectorStore.save(new File(filename));
  }
  // (3) ?t?@?C??????f?[?^??ǂ݂???
  @ShellMethod(key = "vector-load")
  public void load(String filename){
    vectorStore.load(new File(filename));
  }
}

?@SimpleVectorStore?̃C???X?^???X???쐬????ɂ́ASimpleVectorStore.builder???\?b?h??EmbeddingModel???f????n???????ł??i1?j?Bbuild???\?b?h??SimpleVectorStore?̃C???X?^???X???Ԃ???܂??B????ŁA?n???ꂽEmbeddingModel?̓??e??SimpleVectorStore?ő???ł???悤?ɂȂ?܂??B

?@?i2?j?i3?j?́AEmbeddingModel?ŕ\???ꂽ?x?N?g???f?[?^???t?@?C???ɕۑ??A???[?h???邽?߂̃R?[?h?ł??B???X?g4?́Asave???\?b?h?o?R?ŕۑ??????t?@?C???̗?ł??B?????I?ɂ?JSON?`???ŕۑ??????̂ŁAVectorStore???ǂ̂悤?Ƀx?N?g???f?[?^???Ǘ????Ă???̂??A?????ڂɂ??C???[?W???₷???ł??ˁB

{
  "9977d040-28a1-4542-93a8-72002e7534ad" : {
    "text" : "??",
    "embedding" : [ -0.0200862, ..(?ȗ?)...,-0.017477257 ],
    "id" : "9977d040-28a1-4542-93a8-72002e7534ad",
    "metadata" : {
      "category" : "????"
    }
  }
}

VectorStore?ւ̓o?^

?@?????āAVectorStore?ɑ΂??ăx?N?g???f?[?^??o?^???܂??i???X?g5?j?B

@ShellMethod(key = "vector-store")
public String storeText(@ShellOption(value = "--text") String text ,
 @ShellOption(value = "--category" , defaultValue = "general") String category,
 @ShellOption(value = "--tags", defaultValue = "") String tags){
  // (1) ???^?f?[?^(Map)
  Map<String, Object> metadata = new HashMap<>();
  metadata.put("category",category);
  List<String> tagList = List.of(tags.split(","));
  metadata.put("tags",tagList);
  // (2) ?f?[?^?̍쐬
  Document doc = Document.builder()
                         .metadata(metadata).text(text)
                         .build();
  // (3) ?f?[?^?̓o?^
  vectorStore.add(List.of(doc));
  // (?ȗ?)
}

?@?܂??A?i1?j?Ńx?N?g???f?[?^?ɒlj????郁?^?f?[?^??Map?`???ŏ??????܂??B???^?f?[?^?Ƃ??ĊǗ?????L?[?͂ǂ̂悤?Ȃ??̂ł??\???܂??񂪁A?????ł?category??tags??p?ӂ??Ă??܂??B

?@??́A???Ȃ??݂?Builder?p?^?[???Ńh?L???????g?iDocument?j?𐶐????A???^?f?[?^?imetadata?j?ƃe?L?X?g?{?́itext?j?Ƃ?o?^???܂??i2?j?B???̃h?L???????g??add???\?b?h??VectorStore?ɓo?^?ł??܂??i3?j?B

?@?ȏ?𗝉???????A???ۂɎ??s???Ă݂܂??傤?B

shell:>vector-store --text ?U?E?r?[?g???Y --category ?m?y --tags ???y,?C?M???X

VectorStore???g?????ގ??x????

?@????Ńf?[?^?x?[?X?̏??????ł????̂ŁA????????͓o?^?????f?[?^???猟????????@???Љ?܂??B???炩???߁A??قǂ̗????ɑ??̃e?L?X?g?Ȃǂ??F???񎩐g?ł??낢??o?^???Ă????܂??傤?B?܂??A?ȉ??̃R?}???h?????s????Ɠo?^?ς݂̃f?[?^?????p?ł??܂??B

shell:>vector-load sample.json

?@?܂??Љ??̂́A?p??݂̂??w?肵?Č???????A?ł???{?I?ȗ?ł??i???X?g6?j?B

@ShellMethod(key = "vector-search-simple")
public String search(String query){
  // (1) ?????p?̃??N?G?X?g?쐬
  SearchRequest searchRequest =
    SearchRequest.builder()
        .query(query) // ?????p??
        .topK(5) // ?ő?Ŏ擾???錏??(SQL??limit????)
        .build();
  // (2) ?????̎??s
  List<Document> results = vectorStore.similaritySearch(searchRequest);
  // (3) ???????ʂ????Ƀ??X?g?\??
  StringBuffer sb = new StringBuffer();
  results.forEach(doc -> {
    sb.append(String.format("Score : %s , Text : %s , Category : %s\n", doc.getScore(), doc.getText(),doc.getMetadata().get("category")));
  });
  return sb.toString();
}

?@???????N?G?X?g???쐬???Ă???̂́i1?j?ł??B?????p??A?擾????ő匏?????w?肵???̂??Abuild???\?b?h???Ăяo???ASearchRequest?C???X?^???X?𐶐????܂??BSearchRequest?C???X?^???X???????ł??Ă??܂??΁A???similaritySearch???\?b?h?ɂ????n???Č????????s???邾???ł??B

?@?Ȃ??AsimilaritySearch???\?b?h?????s????̂́A???m?ɂ͗ގ??x?????ł??B?i1?j?Ŏw?肵???????p??iquery?j???x?N?g???????ꂽ??ŁA?ގ??x???肳??܂??B

?@???s???ʂ?Document?^?̃??X?g?Ƃ??ĕԂ????̂ŁA?i3?j?ł͂??ꂼ??̃h?L???????g????u??v?x?igetStore?j?A?e?L?X?g?igetText?j?A???^?f?[?^?igetMetadata?j?v?????o???A???X?g?ɐ??`???Ă??܂??B

?@?ȏ?𗝉???????A?????????s???Ă݂܂??傤?B

shell:>vector-search-simple ?r?[?g???Y
Score : 0.8974715547450139 , Text : ?r?[?g???Y , Category : ?m?y
Score : 0.889434231587302 , Text : ?U?E?r?[?g???Y , Category : ?m?y
Score : 0.8469923938240349 , Text : ?r?[?g???W???[?X , Category : ?f??
Score : 0.8336792983536193 , Text : THE BEATLES , Category : ?m?y
Score : 0.8332844177826466 , Text : BEATLES , Category : ?m?y

?@???ʂ͗ގ??x????????????擾?ł??܂??B???̂悤?ȒP???ȒP??ł??A?P????v?????łȂ??A???R?ȗގ??x???猋?ʂ??擾?ł??Ă??邱?Ƃ????Ď??܂??B

???^?f?[?^??p?????????t??????

?@??̌??ʂ????Ă???????悤?ɁA?{???ł???ΑΏۂɂ??Ăق????Ȃ??f?[?^?܂Ō???????A?Ώۂɂ???Ă??܂??Ă??܂??i???̗?ł???΁u?r?[?g???W???[?X?v?j?B???̂悤?ɁA?x?N?g???f?[?^?????ł͖{???Ώۂɂ??????Ȃ??f?[?^?????O????Ȃǂ̏?????????ł????A???̍ۂɂ????^?f?[?^?i?????ł?category?j?𗘗p???邱?ƂŁA???Ӑ}???????ʂ??擾?ł???悤?ɂȂ?܂??i???X?g7?j?B

@ShellMethod(key = "vector-search")
public void searchMetadataString(
  @ShellOption(value = "--query") String query,
  @ShellOption(value = "--category") String category,
  @ShellOption(value = "--threshold", defaultValue = "0.0") String threshold) {
  double doubleThreshold = Double.parseDouble(threshold);
  SearchRequest searchRequest =
    SearchRequest.builder()
        .query(query)
        .filterExpression("category == '" + category + "'") // (1) ?w?肵???J?e?S???Ƃ̈?v???????w?肷??
        .similarityThreshold(doubleThreshold) // (2) ?w?肵???X?R?A?ȏ?
        .topK(5).build();
  //(?ȉ??A?ȗ?) ?????̎??s?????͐?قǂƓ??l
}

?@???^?f?[?^?ɑ΂?????????w?肷??ꍇ?ɂ́A?i1?j?̂悤??SearchRequest.builder??filterExpression???\?b?h???g???Ďw?肵?܂??B?????w??ɂ́A??v?????ȊO?ɂ??A?召?̔?r???Z?q?ł???u>?v??u???t?@?????X???Q?Ƃ??Ă????????B

?@?i2?j?͗ގ??x??臒l?i?????????j??\???܂??B???̗?ł́A?w?肵???X?R?A?ȏ?̒l?݂̂??擾???܂??B????臒l?́A???p????VectorStore?̎?ށi???p????x?N?g???f?[?^?x?[?X?j?₻?̗ގ??x?̌v?Z???W?b?N?Ȃǂɂ???ĕς??̂ŁA???̈Ӗ??ɉ????Đݒ肵?܂??B??????R?T?C???ގ??x?Ƃ??????@???g???Ă??邽?߁A?ő??1?̒l?ŁA1?ɋ߂??قǗގ??x???????Ȃ?܂??B

?@????????s????ƁA?ȉ??̂悤?Ȍ??ʂ??????܂??B

shell:>vector-search --query ?r?[?g???Y --category ?m?y --threshold 0.81
Score : 0.8975537320438945 , Text : ?r?[?g???Y , Category : ?m?y
Score : 0.8895223014114596 , Text : ?U?E?r?[?g???Y , Category : ?m?y
Score : 0.8338045628988857 , Text : THE BEATLES , Category : ?m?y
Score : 0.8334154636622575 , Text : BEATLES , Category : ?m?y

?@?܂??A?????????v???O?????I?ɑg?ݗ??Ă邱?Ƃ??ł??܂??B????ɂ́AFilterExpressionBuilder?𗘗p???Ă????????i???X?g8?j?B

// query = '...' OR tags in ['...'] ?Ɠ??l?̎w??
FilterExpressionBuilder builder = new FilterExpressionBuilder();
var op1 = builder.eq("category",category);
var op2 = builder.in("tags",List.of(tag));
// ?쐬?????I?u?W?F?N?g??filterExpression?ɂĎw??
Filter.Expression expression = builder.or(op1,op2).build();

?܂Ƃ?

?@????́AAI?v???O?????ł悭?g???錟???g???????iRAG?j???x????Z?p?ł???x?N?g???f?[?^?x?[?X?ɂ‚??ďЉ?܂????B?x?N?g???f?[?^?́A????AI???g?????`???b?g?V?X?e???????ł͂Ȃ??A???L?͈͂ɗ??p???₷??????AI?̋@?\???ł????ɗ??p???₷???Z?p?Ƃ?????ł??傤?B

?@????́ARDB???g????VectorStore?ɂ‚??Ă̗??p???@?ɂ‚??ďЉ?܂??B