Repository navigation

Feat/firehose configurable resources - #121

Closed

mabdh wants to merge 142 commits into

raystack:mainfrom

goto:feat/firehose-configurable-resources

Closed

Feat/firehose configurable resources#121
mabdh wants to merge 142 commits into
raystack:mainfrom
goto:feat/firehose-configurable-resources

Commits on Mar 2, 2023
- fix: kube deployment char limit
  spy16
  committed
  ecf98ee
Commits on Mar 7, 2023
- feat: add deployment_id feature
  spy16
  committed
  d48ba77
Commits on Mar 13, 2023
- Merge pull request #1 from goto/fix/bugs
  spy16
  authored
  5d3d7ad
Commits on Mar 14, 2023
- refactor: rename to goto (#2)
  ishanarya0
  authored
  4867aa2
- fix: golanci config (#3)
  ishanarya0
  authored
  3b338aa
Commits on Mar 15, 2023
- fix: kube config handling (#4)
  spy16
  authored
  4d909e6
- fix: deployment id name (#5)
  spy16
  authored
  e96dcf9
Commits on Mar 16, 2023
- Fix/generic (#6)
  spy16
  authored
  202e4ad
- fix: change odpf reference to goto (#7)
  ishanarya0
  authored
  e758108
Commits on Mar 17, 2023
- generic refactors (#8)
  spy16
  authored
  26d5226
- fix: firehose image name (#9)
  spy16
  authored
  ab5b2dc
Commits on Mar 21, 2023
- refactor: re-implement firehose module (#10)
  spy16
  authored
  022f7a4
- fix: use previous telegraf conf (#11)
  spy16
  authored
  4b6a238
- fix: length validation issue (#12)
  spy16
  authored
  152a191
Commits on Mar 24, 2023
- fix: goreleaser commit author (#14)
  ishanarya0
  authored
  f66d8bf
- feat: add stop flag support (#13)
  spy16
  authored
  fade8c7
Commits on Mar 25, 2023
- feat: add stop flag support (#13) (#15)
  spy16
  authored
  226fd7f
Commits on Mar 27, 2023
- fix: revision listing db query (#16)
  spy16
  authored
  cffb11f
- fix: expose chart values type (#17)
  spy16
  authored
  b93cbb8
Commits on Mar 30, 2023
- feat: strip trailing colon in image tag, add telegraf config template
  spy16
  authored
  b063bba
- fix: telegraf config format (#19)
  spy16
  authored
  fb2202d
- feat: add support for dynamic telegraf labels (#20)
  spy16
  authored
  eacb30f
- fix: use go map for telegraf image spec (#21)
  spy16
  authored
  9a37814
- fix: use map to fix yaml marshal (#22)
  spy16
  authored
  52671a6
- fix: set deployment-id length limit as 63 (#23)
  spy16
  authored
  2b799d7
- fix: revert 63 limit to 53 due helm limitation (#24)
  spy16
  authored
  3a368d2
- feat: refactor release name to be consistent with old format (#25)
  spy16
  authored
  d3e42c0
- fix: helm release name limit (#26)
  spy16
  authored
  cbc5e88
Commits on Apr 3, 2023
- feat: implement deployment label templating (#27)
  spy16
  authored
  dc4b7ca
Commits on Apr 4, 2023
- fix: expose name in firehose module (#28)
  spy16
  authored
  199def5
Commits on Apr 5, 2023
- refactor: simplify helm package (#29)
  spy16
  authored
  00fc0d6
- feat: add labels to list-resource api (#30)
  spy16
  authored
  a354d34
Commits on Apr 6, 2023
- feat: merge action and existing labels (#31)
  spy16
  authored
  84ec3cc
- fix: remove name label to prevent length issues (#32)
  spy16
  authored
  4538cb7
Commits on Apr 11, 2023
- feat: add jdbc sink type (#33)
  ishanarya0
  authored
  71766cb
Commits on Apr 12, 2023
- feat: allow custom usage for each firehose (#34)
  ishanarya0
  authored
  4e03452
Commits on Apr 18, 2023
- feat: add tolerations feature (#35)
  
  spy16
  and
  StewartJingga
  authored
  b013688
Commits on Apr 21, 2023
- fix: move volume values outside firehose key (#36)
  
  spy16
  and
  ishanarya0
  authored
  a5f3898
- fix: add secret key (#37)
  ishanarya0
  authored
  d3193e7
Commits on Apr 27, 2023
- feat: add toleration to kube module config (#38)
  ishanarya0
  authored
  0919cc5
Commits on May 9, 2023
- feat: add support for sink specific requests & limits (#39)
  spy16
  authored
  74d17e4
Commits on May 17, 2023
- feat: add firehose base config (#40)
  ishanarya0
  authored
  7e4caa7
Commits on Jul 17, 2023
- feat: offset reset history (#42)
  ishanarya0
  authored
  57a30e8
- feat: add statsd variables to envVariables (#41)
  ishanarya0
  authored
  0128a81
- fix: goreleaser action upgrade (#43)
  ishanarya0
  authored
  e54c6f9
- fix: change rm-dist flag to clean (#44)
  ishanarya0
  authored
  272eb86
- fix: goreleaser (#45)
  ishanarya0
  authored
  1538c02
Commits on Jul 19, 2023
- feat: stoptime in firehose start (#46)
  ishanarya0
  authored
  f3642c1
Commits on Jul 25, 2023
- refactor: public ScaleParams & StartParams (#47)
  ishanarya0
  authored
  cadc134
Commits on Jul 26, 2023
- feat: add urn to deployment labels (#48)
  ishanarya0
  authored
  2217ff1
Commits on Jul 27, 2023
- feat: createdBy and updatedBy in resource & revision (#49)
  ishanarya0
  authored
  178f6a8
- feat: broaden AdditionalGlobalTags scope (#50)
  ishanarya0
  authored
  178e82c
Commits on Jul 28, 2023
- fix: initialize as empty arrays (#51)
  ishanarya0
  authored
  aac1e7f
Commits on Jul 31, 2023
- feat: firehose update to take the latest driver configs (#52)
  ishanarya0
  authored
  0cb3d28
Commits on Aug 7, 2023
- fix: update stopped status (#53)
  ishanarya0
  authored
  b66407d
Commits on Aug 16, 2023
- feat: add logging request body (#54)
  ishanarya0
  authored
  01dc9fe
Commits on Aug 21, 2023
- feat: mounts as secrets (#55)
  ishanarya0
  authored
  b58f67f
- fix: mount secrets (#56)
  ishanarya0
  authored
  43d5983
- fix: key name (#57)
  ishanarya0
  authored
  63bc82b
Commits on Aug 24, 2023
- fix: add bigtable cred path (#58)
  ishanarya0
  authored
  198c661
- fix: remove skipping empty values for env variables (#59)
  ishanarya0
  authored
  e0da338
Commits on Aug 28, 2023
- fix: mount path to be independent of dep id (#60)
  ishanarya0
  authored
  bc5e389
Commits on Aug 30, 2023
- feat: return only running & non-terminating pods (#61)
  ishanarya0
  authored
  4397eb2
- feat: get resource to persist output (#62)
  ishanarya0
  authored
  305fb47
Commits on Aug 31, 2023
- feat: add name as extra label (#64)
  ishanarya0
  authored
  f82209f
Commits on Sep 4, 2023
- feat: support google service account for gke usage (#63)
  mabdh
  authored
  32547ef
Commits on Sep 7, 2023
- test: payloads sent to LOG, BQ and GCS/BLOB sinks (#66)
  ishanarya0
  authored
  851bee4
Commits on Sep 11, 2023
- fix: add delay before executing reset (#69)
  ishanarya0
  authored
  4137fee
Commits on Sep 14, 2023
- fix: job name should be within 63 char limit (#70)
  ishanarya0
  authored
  9773fcd
- fix: remove -firehose from job name (#71)
  ishanarya0
  authored
  ee9a105
Commits on Sep 20, 2023
- feat: old reset strategy (#72)
  ishanarya0
  authored
  2c4ef1f
Commits on Sep 21, 2023
- fix: empty body while logging (#67)
  ishanarya0
  authored
  4fdfd50
Commits on Sep 27, 2023
- feat: add global logger (#75)
  lavkesh
  authored
  876f246
- feat: add global logger (#76)
  lavkesh
  authored
  e6b56cc
Commits on Sep 29, 2023
- feat: adding job module (#74)
  
  lavkesh
  and
  mabdh
  authored
  f0f1a8c
Commits on Oct 3, 2023
- feat: add shared volume (#77)
  lavkesh
  authored
  44b26ce
Commits on Oct 4, 2023
- fead: add sequence if group id does not have it (#73)
  ishanarya0
  authored
  0afa18d
- feat: output refresh and logs for job module (#78)
  lavkesh
  authored
  11a3627
Commits on Oct 6, 2023
- fix: init container issue (#79)
  ishanarya0
  authored
  b0bff0f
Commits on Oct 9, 2023
- feat: on delete cascade in tags table (#80)
  ishanarya0
  authored
  8222c63
Commits on Oct 12, 2023
- feat: add joins in list query (#82)
  ishanarya0
  authored
  ae35b4b
Commits on Oct 13, 2023
- feat: optional spec configs (#83)
  ishanarya0
  authored
  76eba64
Commits on Oct 20, 2023
- fix: allow overiding user passed resource & limits (#84)
  ishanarya0
  authored
  4d0e906
Commits on Nov 2, 2023
- feat: add sink based kube namespace selection (#86)
  ishanarya0
  authored
  2a0d67d
- feat: templatize telegraf config output (#87)
  ishanarya0
  authored
  8579741
Commits on Nov 10, 2023
- fix: remove validation on resource name (#89)
  ishanarya0
  authored
  d0f20ea
Commits on Nov 15, 2023
- chore: if namespace is empty take from default (#85)
  lavkesh
  authored
  f2387ca
Commits on Nov 22, 2023
- Namespace conf (#90)
  lavkesh
  authored
  16ffdac
- Namespace should be taken from resource conf (#91)
  lavkesh
  authored
  3089ca9
Commits on Dec 13, 2023
- feat: make kube deployment timeout, extendInterval etc configurable (#92)
  ishanarya0
  authored
  168a45d
Commits on Feb 23, 2024
- fix: avoid typecheck and upgrade linter version (#94)
  
  manishdangi98
  and
  Manish Dangi
  authored
  69f16e6
- feat: upgrade lint action (#95)
  ishanarya0
  authored
  f77ad4f
- Update .golangci.yml (#96)
  ishanarya0
  authored
  bd86c45
- feat: exclude rules for typecheck linter (#97)
  ishanarya0
  authored
  db0fd9a
- feat(firehose): add HTTPV2 sink type (#93)
  abhishekv24
  authored
  d4ec916
Commits on Mar 7, 2024
- feat: inc writeTimeout (#98)
  ishanarya0
  authored
  b1a2680
Commits on Apr 22, 2024
- feat: helm release config validation during plan (#99)
  ishanarya0
  authored
  35322f1
Commits on May 8, 2024
- feat: list resources pagination (#100)
  FemiNoviaLina
  authored
  e252bc6
Commits on May 14, 2024
- fix: take pagination default value from config (#101)
  FemiNoviaLina
  authored
  ef1cee9
Commits on Jul 11, 2024
- feat: module cli (#102)
  FemiNoviaLina
  authored
  1778229
Commits on Jul 30, 2024
- feat: overridable firehose image repository (#103)
  mabdh
  authored
  136b628
Commits on Aug 13, 2024
- test: e2e-test (#104)
  FemiNoviaLina
  authored
  9077875
Commits on Aug 23, 2024
- feat: modular worker (#105)
  FemiNoviaLina
  authored
  3fcd8c4
Commits on Aug 27, 2024
- fix: change worker config to map (#106)
  FemiNoviaLina
  authored
  d947041
Commits on Aug 28, 2024
- feat: enable opentelemetry for worker (#108)
  FemiNoviaLina
  authored
  417f10e
Commits on Sep 9, 2024
- feat: add kafka resource type (#109)
  FemiNoviaLina
  authored
  80d5637
Commits on Sep 10, 2024
- feat: kafka module config merge (#111)
  FemiNoviaLina
  authored
  eb750e2
Commits on Oct 16, 2024
- feat: add dry run (#116)
  
  ishanarya0
  and
  Ishan Arya
  authored
  bac23e0
Commits on Nov 14, 2024
- feat: allow maxcompute sink (#118)
  FemiNoviaLina
  authored
  46a524e
Commits on Dec 24, 2024
- feat: add dagger (#115)
  
  ishanarya0
  and
  Ishan Arya
  authored
  37c2a81
Commits on Jan 7, 2025
- feat: kube resource support namespace (#121)
  mabdh
  authored
  c124014
Commits on Jan 13, 2025
- fix: flink parallelism update (#122)
  
  ishanarya0
  and
  Ishan Arya
  authored
  77eba57
- Flink parallelism (#123)
  
  ishanarya0
  and
  Ishan Arya
  authored
  6828b8d
Commits on Jan 28, 2025
- fix: fix key to SCHEMA_REGISTRY_STENCIL_URLS (#124)
  
  ishanarya0
  and
  Ishan Arya
  authored
  953741b
Commits on Feb 27, 2025
- feat: add metrics for res status change (#126)
  
  ishanarya0
  and
  Ishan Arya
  authored
  5862a9e
Commits on Feb 28, 2025
- chore: add debug logs (#128)
  
  ishanarya0
  and
  Ishan Arya
  authored
  0fa074c
- chore: counter logs (#129)
  
  ishanarya0
  and
  Ishan Arya
  authored
  163ee15
Commits on Mar 4, 2025
- feat(dagger): add dagger bucket url config (#127)
  FemiNoviaLina
  authored
  eab3919
Commits on Mar 14, 2025
- feat: entropy metrics (#130)
  
  ishanarya0
  and
  Ishan Arya
  authored
  2e13b12
Commits on Apr 18, 2025
- fix: change go struct to go map (#131)
  
  ishanarya0
  and
  Ishan Arya
  authored
  54aaba1
- fix: add empty check (#132)
  
  ishanarya0
  and
  Ishan Arya
  authored
  e882108
Commits on Apr 21, 2025
- feat: add helm max history configuration (#133)
  
  ishanarya0
  and
  Ishan Arya
  authored
  6b32950
Commits on Apr 30, 2025
- fix: add concurrency handling (#134)
  ishanarya0
  authored
  8bc083e
Commits on May 5, 2025
- feat: old reset strategy (#135)
  
  ishanarya0
  and
  Ishan Arya
  authored
  6e6a6fb
Commits on May 15, 2025
- feat: add burst and qps config (#137)
  
  ishanarya0
  and
  Ishan Arya
  authored
  a1f9295
- feat: log failure response (#138)
  
  ishanarya0
  and
  Ishan Arya
  authored
  a9be12e
Commits on Jun 25, 2025
- feat: update flink resource (#139)
  
  ayushi0014
  and
  Ayushi Sharma
  authored
  67d75ea
Commits on Aug 25, 2025
- feat: enable firehose keda autoscaler (#140)
  FemiNoviaLina
  authored
  c85f698
Commits on Aug 26, 2025
- feat(firehose): add keys to keda metadata template config (#141)
  FemiNoviaLina
  authored
  496fc60
Commits on Sep 3, 2025
- feat: kubernetes affinity config (#142)
  FemiNoviaLina
  authored
  4d2cdd5
Commits on Sep 4, 2025
- feat: add desired status in firehose state output (#143)
  mabdh
  authored
  d61dcf9
Commits on Sep 10, 2025
- feat(firehose): support taint and affinity for autoscaler (#144)
  mabdh
  authored
  bdfeda6
- fix: toleration affinity mode kube (#146)
  mabdh
  authored
  7928d5e
Commits on Sep 11, 2025
- feat(firehose): sink based autoscaler config (#145)
  FemiNoviaLina
  authored
  96dac2c
Commits on Sep 15, 2025
- feat(autoscaler): add config validation (#147)
  mabdh
  authored
  995f4d3
Commits on Sep 16, 2025
- feat(firehose): kafka scaler support multiple topic (#148)
  FemiNoviaLina
  authored
  ffa4242
Commits on Sep 17, 2025
- fix: sink based keda config (#149)
  FemiNoviaLina
  authored
  71cfbe6
Commits on Oct 1, 2025
- feat(firehose): allow keda trigger override from resource config (#150)
  FemiNoviaLina
  authored
  47c3a39
Commits on Oct 10, 2025
- feat: add default ttl to remove job after finished (#152)
  
  ayushi0014
  and
  Ayushi Sharma
  authored
  e293ef5
Commits on Oct 13, 2025
- feat: update job ttl to remove job (#153)
  
  ayushi0014
  and
  Ayushi Sharma
  authored
  e5a8465
Commits on Oct 15, 2025
- feat: integrate with otel (#151)
  
  mabdh
  and
  FemiNoviaLina
  authored
  168093e
Commits on Jul 7, 2026
- feat(firehose): declare limits/requests in config schema
  
  mabdh
  and
  claude
  committed
  672bb69

Commits

Commits on Mar 2, 2023

Commits on Mar 7, 2023

Commits on Mar 13, 2023

Commits on Mar 14, 2023

Commits on Mar 15, 2023

Commits on Mar 16, 2023

Commits on Mar 17, 2023

Commits on Mar 21, 2023

Commits on Mar 24, 2023

Commits on Mar 25, 2023

Commits on Mar 27, 2023

Commits on Mar 30, 2023

Commits on Apr 3, 2023

Commits on Apr 4, 2023

Commits on Apr 5, 2023

Commits on Apr 6, 2023

Commits on Apr 11, 2023

Commits on Apr 12, 2023

Commits on Apr 18, 2023

Commits on Apr 21, 2023

Commits on Apr 27, 2023

Commits on May 9, 2023

Commits on May 17, 2023

Commits on Jul 17, 2023

Commits on Jul 19, 2023

Commits on Jul 25, 2023

Commits on Jul 26, 2023

Commits on Jul 27, 2023

Commits on Jul 28, 2023

Commits on Jul 31, 2023

Commits on Aug 7, 2023

Commits on Aug 16, 2023

Commits on Aug 21, 2023

Commits on Aug 24, 2023

Commits on Aug 28, 2023

Commits on Aug 30, 2023

Commits on Aug 31, 2023

Commits on Sep 4, 2023

Commits on Sep 7, 2023

Commits on Sep 11, 2023

Commits on Sep 14, 2023

Commits on Sep 20, 2023

Commits on Sep 21, 2023

Commits on Sep 27, 2023

Commits on Sep 29, 2023

Commits on Oct 3, 2023

Commits on Oct 4, 2023

Commits on Oct 6, 2023

Commits on Oct 9, 2023

Commits on Oct 12, 2023

Commits on Oct 13, 2023

Commits on Oct 20, 2023

Commits on Nov 2, 2023

Commits on Nov 10, 2023

Commits on Nov 15, 2023

Commits on Nov 22, 2023