Skip to content

Commit 5f5707c

Browse files
authored
Merge branch 'master' into dev_op_var_len_expand
2 parents 6a986b7 + bd977ee commit 5f5707c

34 files changed

+1778
-47
lines changed

docs/en-US/source/14.faq.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,33 +3,56 @@
33
During the use of TuGraph, if you encounter problems, you can first check [User Documentation](https://github.com/TuGraph-db/tugraph-db/tree/master/doc/zh-CN). Second, check to see if the FAQ list below matches your question.
44

55
1. **[Kernel Engine QA Summary](https://github.com/TuGraph-db/tugraph-db/discussions/113#discussion-4575385)**
6+
67
1.1 [Does the edge of TuGraph support index? ](https://github.com/TuGraph-db/tugraph-db/discussions/113#discussioncomment-4165379)
8+
79
1.2 [What is the QPS of a single TuGraph machine? ](https://github.com/TuGraph-db/tugraph-db/discussions/113#discussioncomment-4165388)
810
2. **[TuGraph Browser QA Summary](https://github.com/TuGraph-db/tugraph-db/discussions/111#discussion-4575381)**
11+
912
2.1 [How to update the tugraph service after the visualization file is built? ](https://github.com/TuGraph-db/tugraph-db/discussions/111#discussioncomment-4165364)
13+
1014
2.2 [How to connect the existing tugraph service through npm run dev? ](https://github.com/TuGraph-db/tugraph-db/discussions/111#discussioncomment-4165366)
15+
1116
2.3 [TuGraph Browser version 3.3.0 failed to delete node label? ](https://github.com/TuGraph-db/tugraph-db/discussions/82#discussion-4465791)
1217
3. **[Client QA Summary](https://github.com/TuGraph-db/tugraph-db/discussions/109#discussion-4575378)**
18+
1319
3.1 [What programming languages ​​does the client currently have, and does it support node js? ](https://github.com/TuGraph-db/tugraph-db/discussions/109#discussioncomment-4165351)
20+
1421
3.2 [Does the python client support pip install? Where is the client referenced? ](https://github.com/TuGraph-db/tugraph-db/discussions/109#discussioncomment-4165353)
1522
4. **[Data Import QA Summary](https://github.com/TuGraph-db/tugraph-db/discussions/107#discussion-4575373)**
23+
1624
4.1 [Which common databases can TuGraph connect to? ](https://github.com/TuGraph-db/tugraph-db/discussions/107#discussioncomment-4165335)
25+
1726
4.2 [How to import data? ](https://github.com/TuGraph-db/tugraph-db/discussions/107#discussioncomment-4165336)
1827
5. **[Stored Procedure QA Summary](https://github.com/TuGraph-db/tugraph-db/discussions/105#discussion-4575369)**
28+
1929
5.1 [How to load stored procedure or algorithm package? ](https://github.com/TuGraph-db/tugraph-db/discussions/105#discussioncomment-4165313)
30+
2031
5.2 [How to call or execute the stored procedure? ](https://github.com/TuGraph-db/tugraph-db/discussions/105#discussioncomment-4165317)
32+
2133
5.3 [Where is the open source built-in algorithm package? ](https://github.com/TuGraph-db/tugraph-db/discussions/105#discussioncomment-4165322)
2234
6. **[Installation and deployment QA summary](https://github.com/TuGraph-db/tugraph-db/discussions/103#discussion-4575364)**
35+
2336
6.1 [How to install using docker image? ](https://github.com/TuGraph-db/tugraph-db/discussions/103#discussioncomment-4165287)
37+
2438
6.2 [After the rpm package and deb package are installed, start the lgraph_server service. Prompt missing 'liblgraph.so' error? ](https://github.com/TuGraph-db/tugraph-db/discussions/103#discussioncomment-4165289)
39+
2540
6.3 [How to compile TuGraph on Mac's M1 chip? ](https://github.com/TuGraph-db/tugraph-db/discussions/47#discussion-4393165)
41+
42+
6.4 [How to Handle Error: 'jemalloc: Unsupported system page size'? ](https://github.com/TuGraph-family/tugraph-db/issues/477)
2643
7. **[Cypher QA Summary](https://github.com/TuGraph-db/tugraph-db/discussions/102#discussion-4575018)**
44+
2745
7.1 [Does it support conditional query of indefinite length? ](https://github.com/TuGraph-db/tugraph-db/discussions/102#discussioncomment-4165252)
46+
2847
7.2 [How to query the shortest path, how to use the shortestPath function? ](https://github.com/TuGraph-db/tugraph-db/discussions/102#discussioncomment-4165256)
48+
2949
7.3 [Using and after the query statement Where to splice the query speed is slow, how should the statement be optimized and improved? ](https://github.com/TuGraph-db/tugraph-db/discussions/102#discussioncomment-4165260)
50+
3051
7.4 [How to query the edge of any jump? ](https://github.com/TuGraph-db/tugraph-db/discussions/102#discussioncomment-4165262)
52+
3153
7.5 [Is there a date() function in Cypher? ](https://github.com/TuGraph-db/tugraph-db/discussions/91#discussion-4482858)
3254
8. **[Jwt Token QA Summay](https://github.com/TuGraph-family/tugraph-db/discussions/282#discussion-5574402)**
55+
3356
8.1 [What should I do after the error "User has reached the maximum number of tokens" is reported? ](https://github.com/TuGraph-family/tugraph-db/discussions/282#discussioncomment-6861785)
3457

3558
If the problem is still not resolved, we recommend you follow up or create a new thread describing your problem. At the same time, the TuGraph QA summary will be updated regularly based on the posts initiated by everyone in the discussion area.

docs/zh-CN/source/14.faq.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@
3939
6.2 [rpm 包和 deb 包安装后,启动 lgraph_server 服务,提示缺少'liblgraph.so'报错?](https://github.com/TuGraph-family/tugraph-db/discussions/103#discussioncomment-4165289)
4040

4141
6.3 [如何在 Mac 的 M1 芯片上编译 TuGraph?](https://github.com/TuGraph-family/tugraph-db/discussions/47#discussion-4393165)
42+
43+
6.4 [安装报错 jemalloc: Unsupported system page size 该如何处理?](https://github.com/TuGraph-family/tugraph-db/issues/477)
4244
7. **[Cypher QA 汇总](https://github.com/TuGraph-family/tugraph-db/discussions/102#discussion-4575018)**
4345

4446
7.1 [是否支持不定长边的条件查询?](https://github.com/TuGraph-family/tugraph-db/discussions/102#discussioncomment-4165252)

include/lgraph/lgraph_exceptions.h

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,8 @@ X(TxnCommitException, "Txn commit exception.") \
7575
X(ReminderException, "Reminder exception.") \
7676
X(GraphCreateException, "Graph create exception.") \
7777
X(CypherParameterTypeError, "Cypher parameter type error.") \
78-
X(ReachMaximumEid, "Edge eid exceeds the limit.")
78+
X(ReachMaximumEid, "Edge eid exceeds the limit.") \
79+
X(ReachMaximumCompositeIndexField, "The size of composite index fields exceeds the limit.")
7980

8081
enum class ErrorCode {
8182
#define X(code, msg) code,

include/lgraph/lgraph_txn.h

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -289,6 +289,24 @@ class Transaction {
289289
int64_t AddVertex(size_t label_id, const std::vector<size_t>& field_ids,
290290
const std::vector<FieldData>& field_values);
291291

292+
/**
293+
* @brief Upsert a vertex.
294+
*
295+
* @param label_id Label id.
296+
* @param primary_pos The location of the primary field in field_ids.
297+
* @param unique_pos The locations of the unique index field in field_ids, can be empty.
298+
* @param field_ids List of field ids.
299+
* @param field_values The field values.
300+
*
301+
* @returns 0: nothing happened because of index conflict
302+
* 1: the vertex is inserted
303+
* 2: the vertex is updated
304+
*/
305+
int UpsertVertex(size_t label_id, size_t primary_pos,
306+
const std::vector<size_t>& unique_pos,
307+
const std::vector<size_t>& field_ids,
308+
const std::vector<FieldData>& field_values);
309+
292310
/**
293311
* @brief Adds an edge. All non-nullable fields must be specified. An exception is thrown
294312
* if src or dst does not exist.
@@ -386,6 +404,26 @@ class Transaction {
386404
bool UpsertEdge(int64_t src, int64_t dst, size_t label_id, const std::vector<size_t>& field_ids,
387405
const std::vector<FieldData>& field_values);
388406

407+
/**
408+
* @brief Upsert edge. If there is no src->dst edge, insert it. Otherwise, try to update
409+
* the edge's property.
410+
*
411+
* @param src Source vertex id.
412+
* @param dst Destination vertex id.
413+
* @param label_id The label id.
414+
* @param unique_pos The locations of the unique index field in field_ids, can be empty.
415+
* @param field_ids List of field ids.
416+
* @param field_values List of field values.
417+
*
418+
* @returns 0: nothing happened because of index conflict
419+
* 1: the vertex is inserted
420+
* 2: the vertex is updated
421+
*/
422+
int UpsertEdge(int64_t src, int64_t dst, size_t label_id,
423+
const std::vector<size_t>& unique_pos,
424+
const std::vector<size_t>& field_ids,
425+
const std::vector<FieldData>& field_values);
426+
389427
/**
390428
* @brief List indexes
391429
*

include/lgraph/lgraph_types.h

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1208,6 +1208,13 @@ enum class IndexType {
12081208
PairUniqueIndex = 2
12091209
};
12101210

1211+
enum class CompositeIndexType {
1212+
/** @brief this is not unique composite index
1213+
* Temporarily require all attributes to be non-empty attributes
1214+
* */
1215+
UniqueIndex = 1
1216+
};
1217+
12111218
/** @brief An index specifier. */
12121219
struct IndexSpec {
12131220
/** @brief label name */
@@ -1243,6 +1250,10 @@ struct EdgeUid {
12431250
tid == rhs.tid;
12441251
}
12451252

1253+
inline bool operator!=(const EdgeUid& rhs) const {
1254+
return !this->operator==(rhs);
1255+
}
1256+
12461257
inline bool operator<(const EdgeUid& rhs) const {
12471258
return src < rhs.src || (src == rhs.src && dst < rhs.dst) ||
12481259
(src == rhs.src && dst == rhs.dst && lid < rhs.lid) ||

src/BuildLGraphApi.cmake

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,8 @@ set(LGRAPH_CORE_SRC
5050
core/vertex_index.cpp
5151
core/wal.cpp
5252
core/lmdb/mdb.c
53-
core/lmdb/midl.c)
53+
core/lmdb/midl.c
54+
core/composite_index.cpp)
5455

5556
set(LGRAPH_DB_SRC
5657
db/acl.cpp

src/core/composite_index.cpp

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
/**
2+
* Copyright 2024 AntGroup CO., Ltd.
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
*/
14+
15+
#include "core/composite_index.h"
16+
#include "core/transaction.h"
17+
18+
namespace lgraph {
19+
20+
Value CompositeIndexValue::CreateKey(const Value &key) const {
21+
int pos = GetVidCount() - 1;
22+
Value v(key.Size() + _detail::VID_SIZE);
23+
memcpy(v.Data(), key.Data(), key.Size());
24+
memcpy(v.Data() + key.Size(), v_.Data() + 1 + pos * _detail::VID_SIZE, _detail::VID_SIZE);
25+
return v;
26+
}
27+
28+
CompositeIndex::CompositeIndex(std::shared_ptr<KvTable> table, std::vector<FieldType> key_types,
29+
CompositeIndexType type) : table_(std::move(table)), key_types(std::move(key_types)),
30+
ready_(false), disabled_(false), type_(type) {}
31+
32+
CompositeIndex::CompositeIndex(const CompositeIndex& rhs)
33+
: table_(rhs.table_),
34+
key_types(rhs.key_types),
35+
ready_(rhs.ready_.load()),
36+
disabled_(rhs.disabled_.load()),
37+
type_(rhs.type_) {}
38+
39+
std::unique_ptr<KvTable> CompositeIndex::OpenTable(KvTransaction &txn, KvStore &store,
40+
const std::string &name,
41+
const std::vector<FieldType> &dt,
42+
CompositeIndexType type) {
43+
ComparatorDesc desc;
44+
switch (type) {
45+
case CompositeIndexType::UniqueIndex:
46+
{
47+
desc.comp_type = ComparatorDesc::COMPOSITE_KEY;
48+
break;
49+
}
50+
}
51+
desc.data_types = dt;
52+
return store.OpenTable(txn, name, true, desc);
53+
}
54+
55+
void CompositeIndex::_AppendCompositeIndexEntry(KvTransaction& txn, const Value& k, VertexId vid) {
56+
FMA_DBG_ASSERT(type_ == CompositeIndexType::UniqueIndex);
57+
if (k.Size() >= _detail::MAX_KEY_SIZE) {
58+
txn.Abort();
59+
THROW_CODE(ReachMaximumCompositeIndexField, "The key of the composite index is "
60+
"too long and exceeds the limit.");
61+
}
62+
table_->AppendKv(txn, k, Value::ConstRef(vid));
63+
}
64+
65+
bool CompositeIndex::Add(KvTransaction& txn, const Value& k, int64_t vid) {
66+
if (k.Size() >= _detail::MAX_KEY_SIZE) {
67+
txn.Abort();
68+
THROW_CODE(ReachMaximumCompositeIndexField, "The key of the composite index is "
69+
"too long and exceeds the limit.");
70+
}
71+
switch (type_) {
72+
case CompositeIndexType::UniqueIndex:
73+
{
74+
return table_->AddKV(txn, k, Value::ConstRef(vid));
75+
}
76+
}
77+
return false;
78+
}
79+
80+
} // namespace lgraph

src/core/composite_index.h

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
/**
2+
* Copyright 2024 AntGroup CO., Ltd.
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
*/
14+
15+
#pragma once
16+
17+
#include <atomic>
18+
#include <exception>
19+
#include <unordered_map>
20+
21+
#include "core/field_data_helper.h"
22+
#include "core/data_type.h"
23+
#include "core/iterator_base.h"
24+
#include "core/kv_store.h"
25+
#include "core/kv_table_comparators.h"
26+
#include "core/type_convert.h"
27+
#include "core/value.h"
28+
29+
namespace lgraph {
30+
31+
class CompositeIndexValue {
32+
friend class CompositeIndex;
33+
34+
Value v_;
35+
36+
CompositeIndexValue(): v_(1) { *(uint8_t*)v_.Data() = 0; }
37+
38+
explicit CompositeIndexValue(const Value& v): v_(v) {}
39+
40+
explicit CompositeIndexValue(Value&& v): v_(std::move(v)) {}
41+
42+
int GetVidCount() const { return static_cast<int>(*(uint8_t*)v_.Data()); }
43+
44+
template <typename IT>
45+
CompositeIndexValue(const IT& beg, const IT& end) {
46+
size_t n = end - beg;
47+
FMA_DBG_ASSERT(n < std::numeric_limits<uint8_t>::max());
48+
v_.Resize(1 + _detail::VID_SIZE * n);
49+
char* p = v_.Data();
50+
*(uint8_t*)p = (uint8_t)n;
51+
p++;
52+
for (auto it = beg; it < end; it++) {
53+
_detail::WriteVid(p, *it);
54+
p += _detail::VID_SIZE;
55+
}
56+
}
57+
58+
/**
59+
* Patch the key with the last vid in this VertexCompositeIndexValue.
60+
*/
61+
Value CreateKey(const Value& key) const;
62+
63+
const Value& GetBuf() const { return v_; }
64+
65+
Value& GetBuf() { return v_; }
66+
};
67+
68+
/**
69+
* The indices
70+
*/
71+
class CompositeIndex {
72+
friend class LightningGraph;
73+
74+
std::shared_ptr<KvTable> table_;
75+
std::vector<FieldType> key_types;
76+
std::atomic<bool> ready_;
77+
std::atomic<bool> disabled_;
78+
CompositeIndexType type_;
79+
80+
public:
81+
CompositeIndex(std::shared_ptr<KvTable> table, std::vector<FieldType> key_types,
82+
CompositeIndexType type);
83+
84+
CompositeIndex(const CompositeIndex& rhs);
85+
86+
CompositeIndex(CompositeIndex&& rhs) = delete;
87+
88+
CompositeIndex& operator=(const CompositeIndex& rhs) = delete;
89+
90+
CompositeIndex& operator=(CompositeIndex&& rhs) = delete;
91+
92+
static std::unique_ptr<KvTable> OpenTable(KvTransaction& txn, KvStore& store,
93+
const std::string& name,
94+
const std::vector<FieldType> &dt,
95+
CompositeIndexType type);
96+
97+
void _AppendCompositeIndexEntry(KvTransaction& txn, const Value& k, VertexId vid);
98+
99+
bool Add(KvTransaction& txn, const Value& k, int64_t vid);
100+
101+
private:
102+
void Clear(KvTransaction& txn) { table_->Drop(txn); }
103+
104+
void SetReady() { ready_.store(true, std::memory_order_release); }
105+
106+
void Disable() { disabled_.store(true, std::memory_order_release); }
107+
108+
void Enable() { disabled_.store(false, std::memory_order_release); }
109+
110+
[[nodiscard]] bool IsDisabled() const { return disabled_.load(std::memory_order_acquire); }
111+
};
112+
} // namespace lgraph

src/core/data_type.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ typedef lgraph_api::FieldType FieldType;
3737
typedef lgraph_api::FieldData FieldData;
3838
typedef lgraph_api::IndexType IndexType;
3939
typedef lgraph_api::IndexSpec IndexSpec;
40+
typedef lgraph_api::CompositeIndexType CompositeIndexType;
4041
typedef lgraph_api::FieldSpec FieldSpec;
4142
typedef lgraph_api::EdgeUid EdgeUid;
4243
typedef lgraph_api::Date Date;

src/core/defs.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,7 @@ static const size_t MAX_LABEL_NAME_LEN = 256;
158158
static const size_t MAX_FIELD_NAME_LEN = 256;
159159
static const size_t MAX_PASSWORD_LEN = 64;
160160
static const size_t MAX_DESC_LEN = 512;
161+
static const size_t MAX_COMPOSITE_FILED_SIZE = 16;
161162

162163
static const size_t DEFAULT_MEM_LIMIT = (size_t)2 << 40;
163164
} // namespace _detail

0 commit comments

Comments
 (0)