pre-commit config top level exclude doesn't work? - mypy

I have the following .pre-commit-config.yaml file. At the first line, I have this top level exclude path to exclude all checks in that folder, but for some reason, the mypy hook still outputs errors from files in that folder. Could someone help me understand what's going on? Thanks.
exclude: "src/project/proto/"
default_language_version:
python: python3.8
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-ast
- id: check-docstring-first
- id: check-executables-have-shebangs
- id: check-json
- id: check-merge-conflict
- id: check-yaml
- id: debug-statements
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/asottile/pyupgrade
rev: v3.2.2
hooks:
- id: pyupgrade
- repo: https://github.com/asottile/reorder_python_imports
rev: v3.9.0
hooks:
- id: reorder-python-imports
args: [--py3-plus]
- repo: https://github.com/asottile/add-trailing-comma
rev: v2.3.0
hooks:
- id: add-trailing-comma
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.991
hooks:
- id: mypy
args: [--strict]
additional_dependencies:
[types-protobuf>=4.21.0.0, types-termcolor>=1.1.6, pytest>=7.2.0]
- repo: https://github.com/PyCQA/flake8
rev: 6.0.0
hooks:
- id: flake8
I also have this mypy.ini. Just in case, I missed anything here.
[mypy]
python_version = 3.8
mypy_path = ./src:./tests
[mypy-lark.*]
ignore_missing_imports = True

it is working, but not in the way you expect
mypy performs dynamic analysis and will follow imports. pre-commit is not passing those filenames onto mypy but they are being checked due to follow-imports
you will need to also use mypy's settings to ignore errors there
disclaimer: I created pre-commit

Related

How to specify a similiarities options checker of pylint in pre-commit args?

I've been trying to configure my pre-commit config file to ignore similiarities on imports and docstrings as described in: https://pylint.pycqa.org/en/latest/technical_reference/features.html?highlight=similar#similarities-checker-options
But I don't use any of .pylintrc files, just the .pre-commit-config.yaml.
Below is a snippet from .pre-commit-config.yaml
- id: pylint
name: pylint
entry: pylint
language: python
require_serial: true
types_or: [python, pyi]
exclude: 'kedro-init'
args: ['--disable=E0401,E1101,E1102,R0913,R0914,W0703,E0602,C0103,C0114,
W0102,C0330,C0326,W0107,R1716,R0902,E0611,E1124', '--fail-under=7.5',
'--ignore-imports=yes', '--ignore-docstrings=yes'] #this last line does not work
Is there any way to specify those options on the args key?
Thx!
python version: 3.8.x
pylint version: 2.12.3
If you retype your args in .pre-commit-config.yaml as below, it should work.
- repo: local
hooks:
- id: pylint
name: pylint
entry: pylint
language: python
require_serial: true
types_or: [python, pyi]
exclude: 'kedro-init'
args: ["--ignore=E0401,E1101,E1102,R0913,R0914,W0703,E0602,C0103,
C0114, W0102,C0330,C0326,W0107,R1716,R0902,E0611,E1124",
"--fail-under=7.5"]

Swift Package Manager - binaryTarget with .zip file fails to validate

I tried different approaches to add a binaryTarget to a Swift package - 2 of them worked out fine (Target1 and Target2 in the example), but third approach (Target3) that should also work according to documentation does not validate: unsupported extension for binary target 'Target3'; valid extensions are: xcframework
For not bloating the repo too much with every binary release I would prefer the zip approach here... - Anyone got it working with a binaryTarget and a .zip file in path: added to the Package repository, or any hints what I'm doing wrong here?
(Xcode 12.4, t3.zip containing only the .xcframework at root level)
// swift-tools-version:5.3
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "StackoverflowExamplePackage",
platforms: [
.iOS(.v9)
],
products: [
.library(
name: "Lib1",
targets: ["Target1"]),
.library(
name: "Lib2",
targets: ["Target2", "Target3"]),
],
dependencies: [
// .package(url: /* package url */, from: "1.0.0"),
],
targets: [
.binaryTarget(
name: "Target1",
url: "https://myurl.example.com/t1-xcframework.zip",
checksum: "777ddd6381e2201b7eb778b72f373f77e1190fd9dc9503f703e37c86d3b89674"
),
.binaryTarget(name: "Target2", path: "./Binaries/t2.xcframework"),
.binaryTarget(name: "Target3", path: "./Binaries/t3.zip"),
]
)
Zip archive support for local binary targets in SPM was merged last year in October and has been finally released along with Xcode 13.3.

Missing bundle entity, entity type paragraphs_type from field_instance.yml file

I'm currently trying to migrate my D7 site to D8 and one of the things migrating is field_collections to paragraphs. I have a field_collection call "field_sidebar_callouts". I want to migrate that to paragraphs.
Everything is migrating correctly until it reaches to field_instance.yml file. It throws an error call this:
Missing bundle entity, entity type paragraphs_type, entity id r_callouts
Is the instance yml file importing half the id? How come the entity id is only "r_callouts" and not "field_sidebar_callouts".
Did my field_instance.yml get auto-generated incorrectly?
Any help would be amazing!
Here's my instance file:
uuid: 295c39a9-1857-4d53-a342-92a16ae71223
langcode: en
status: true
dependencies: { }
id: upgrade_d7_field_instance
class: Drupal\migrate_drupal\Plugin\migrate\FieldMigration
field_plugin_method: alterFieldInstanceMigration
cck_plugin_method: null
migration_tags:
- 'Drupal 7'
- Configuration
migration_group: migrate_drupal_7
label: 'Field instance configuration'
source:
plugin: d7_field_instance
constants:
status: true
process:
type:
-
plugin: process_field
source: type
method: getFieldType
entity_type:
-
plugin: get
source: entity_type
-
plugin: static_map
map:
field_collection_item: paragraph
paragraphs_item: paragraph
bypass: true
field_name:
-
plugin: get
source: field_name
bundle:
-
plugin: static_map
source: bundle
bypass: true
map:
comment_node_forum: comment_forum
-
plugin: paragraphs_process_on_value
source_value: entity_type
expected_value: field_collection_item
process:
plugin: substr
start: 6
label:
-
plugin: get
source: label
description:
-
plugin: get
source: description
required:
-
plugin: get
source: required
status:
-
plugin: get
source: constants/status
allowed_values:
-
plugin: sub_process
source: allowed_vid
process:
-
plugin: migration_lookup
migration: upgrade_d7_taxonomy_vocabulary
source: vid
settings:
0:
plugin: d7_field_instance_settings
source:
- settings
- widget
- field_definition
field_collection:
plugin: field_collection_field_instance_settings
default_value_function:
-
plugin: get
source: ''
default_value:
-
plugin: d7_field_instance_defaults
source:
- default_value
- widget
translatable:
-
plugin: get
source: translatable
settings/title:
-
plugin: static_map
source: settings/title
bypass: true
map:
disabled: 0
optional: 1
required: 2
destination:
plugin: 'entity:field_config'
migration_dependencies:
required:
- upgrade_d7_field
optional:
- upgrade_d7_node_type
- upgrade_d7_comment_type
- upgrade_d7_taxonomy_vocabulary
- upgrade_d7_field_collection_type

Not able to execute lifecycle operation using script plugin

I'm trying to learn how to use script plugin. I'm following script plugin docs here but not able to make it work.
I've tried to use the plugin in two ways. The first, when cloudify.interface.lifecycle.start operation is mapped directly to a script:
tosca_definitions_version: cloudify_dsl_1_3
imports:
- 'http://www.getcloudify.org/spec/cloudify/4.5.5/types.yaml'
node_templates:
Import_Project:
type: cloudify.nodes.WebServer
capabilities:
scalable:
properties:
default_instances: 1
interfaces:
cloudify.interfaces.lifecycle:
start:
implementation: scripts/create_project.sh
inputs: {}
The second with a direct mapping:
tosca_definitions_version: cloudify_dsl_1_3
imports:
- 'http://www.getcloudify.org/spec/cloudify/4.5.5/types.yaml'
node_templates:
Import_Project:
type: cloudify.nodes.WebServer
capabilities:
scalable:
properties:
default_instances: 1
interfaces:
cloudify.interfaces.lifecycle:
start:
implementation: script.script_runner.tasks.run
inputs:
script_path: scripts/create_project.sh
I've created a directory named scripts and placed the below create_project.sh script in this directory:
#! /bin/bash -e
ctx logger info "Hello to this world"
hostname
I'm getting errors while validating the blueprint.
Error when operation is mapped directly to a script:
[2019-04-13 13:29:40.594] [DEBUG] DslParserExecClient - got output from dsl parser Could not extract plugin from operation mapping 'scripts/create_project.sh', which is declared for operation 'start'. In interface 'cloudify.interfaces.lifecycle' in node 'Import_Project' of type 'cloudify.nodes.WebServer'
in: /opt/cloudify-composer/backend/dev/workspace/2/tmp-27O0e1t813N6as
in line: 3, column: 2
path: node_templates.Import_Project
value: {'interfaces': {'cloudify.interfaces.lifecycle': {'start': {'implementation': 'scripts/create_project.sh', 'inputs': {}}}}, 'type': 'cloudify.nodes.WebServer', 'capabilities': {'scalable': {'properties': {'default_instances': 1}}}}
Error when using a direct mapping:
[2019-04-13 13:25:21.015] [DEBUG] DslParserExecClient - got output from dsl parser node 'Import_Project' has no relationship which makes it contained within a host and it has a plugin 'script' with 'host_agent' as an executor. These types of plugins must be installed on a host
in: /opt/cloudify-composer/backend/dev/workspace/2/tmp-279QCz2CV3Y81L
in line: 2, column: 0
path: node_templates
value: {'Import_Project': {'interfaces': {'cloudify.interfaces.lifecycle': {'start': {'implementation': 'script.script_runner.tasks.run', 'inputs': {'script_path': 'scripts/create_project.sh'}}}}, 'type': 'cloudify.nodes.WebServer', 'capabilities': {'scalable': {'properties': {'default_instances': 1}}}}}
What is missing to make this work?
I also found the Cloudify Script Plugin examples from their documentation do not work: https://docs.cloudify.co/4.6/working_with/official_plugins/configuration/script/
However, I found I can make the examples work by adding an executor line in parallel with the implementation line to override the host_agent executor as follows:
tosca_definitions_version: cloudify_dsl_1_3
imports:
- 'http://www.getcloudify.org/spec/cloudify/4.5.5/types.yaml'
node_templates:
Import_Project:
type: cloudify.nodes.WebServer
capabilities:
scalable:
properties:
default_instances: 1
interfaces:
cloudify.interfaces.lifecycle:
start:
implementation: scripts/create_project.sh
executor: central_deployment_agent
inputs: {}

How can I found that the Rscript have been run successfully on the docker by the CWL?

I have written the CWL code which runs the Rscript command in the docker container. I have two files of cwl and yaml and run them by the command:
cwltool --debug a_code.cwl a_input.yaml
I get the output which said that the process status is success but there is no output file and in the result "output": null is reported. I want to know if there is a method to find that the Rscript file has run on the docker successfully. I want actually know about the reason that the output files are null.
The final part of the result is:
[job a_code.cwl] {
"output": null,
"errorFile": null
}
[job a_code.cwl] Removing input staging directory /tmp/tmpUbyb7k
[job a_code.cwl] Removing input staging directory /tmp/tmpUbyb7k
[job a_code.cwl] Removing temporary directory /tmp/tmpkUIOnw
[job a_code.cwl] Removing temporary directory /tmp/tmpkUIOnw
Removing intermediate output directory /tmp/tmpCG9Xs1
Removing intermediate output directory /tmp/tmpCG9Xs1
{
"output": null,
"errorFile": null
}
Final process status is success
Final process status is success
R code:
library(cummeRbund)
cuff<-readCufflinks( dbFile = "cuffData.db", gtfFile = NULL, runInfoFile = "run.info", repTableFile = "read_groups.info", geneFPKM = "genes.fpkm_trac .... )
#setwd("/scripts")
sink("cuff.txt")
print(cuff)
sink()
My cwl file code is:
class: CommandLineTool
cwlVersion: v1.0
id: cummerbund
baseCommand:
- Rscript
inputs:
- id: Rfile
type: File?
inputBinding:
position: 0
- id: cuffdiffout
type: 'File[]?'
inputBinding:
position: 1
- id: errorout
type: File?
inputBinding:
position: 99
prefix: 2>
valueFrom: |
error.txt
outputs:
- id: output
type: File?
outputBinding:
glob: cuff.txt
- id: errorFile
type: File?
outputBinding:
glob: error.txt
label: cummerbund
requirements:
- class: DockerRequirement
dockerPull: cummerbund_0
my input file (yaml file) is:
inputs:
Rfile:
basename: run_cummeR.R
class: File
nameext: .R
nameroot: run_cummeR
path: run_cummeR.R
cuffdiffout:
- class: File
path: cuffData.db
- class: File
path: genes.fpkm_tracking
- class: File
path: read_groups.info
- class: File
path: genes.count_tracking
- class: File
path: genes.read_group_tracking
- class: File
path: isoforms.fpkm_tracking
- class: File
path: isoforms.read_group_tracking
- class: File
path: isoforms.count_tracking
- class: File
path: isoform_exp.diff
- class: File
path: gene_exp.diff
errorout:
- class: File
path: error.txt
Also, this is my Dockerfile for creating image:
FROM r-base
COPY . /scripts
RUN apt-get update
RUN apt-get install -y \
libcurl4-openssl-dev\
libssl-dev\
libmariadb-client-lgpl-dev\
libmariadbclient-dev\
libxml2-dev\
r-cran-plyr\
r-cran-reshape2
WORKDIR /scripts
RUN Rscript /scripts/build.R
ENTRYPOINT /bin/bash
I got the answer!
There were some problems in my program.
1. The docker was not pulled correctly then the cwl couldn't produce any output.
2. The inputs and outputs were not defined mandatory. So I got the success status in the case which I did not have proper inputs and output.

Resources